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FOREWORD 


This  document  is  a  description  of  the  research  activities  conducted  during  the 
fourth  year  of  the  Building  the  Career  Force  project.  This  project  is  the  second  phase 
of  a  research  program  of  unprecedented  scope  and  depth  to  provide  the  basis  for 
improving  the  Army’s  selection  and  classification  procedures,  as  well  as  improving 
reenlistment  and  promotion  decisions  for  soldiers  up  to  the  level  of  sergeant.  The 
thrust  for  this  program  came  from  the  practical,  professional,  and  legal  need  to  validate 
the  Armed  Services  Vocational  Aptitude  Battery  (ASVAB),  the  current  U.S.  military 
selection/classification  test  battery,  and  other  selection  variables  as  predictors  of 
training  and  performance.  The  authorization  for  the  program  was  provided  in  a  letter, 
Deputy  Chief  of  Staff  for  Operations,  “Army  Research  Project  to  Validate  the  Predictive 
Value  of  the  Armed  Services  Vocational  Aptitude  Batter,”  effective  1 9  November  1 980, 
and  a  memorandum,  Assistant  Secretary  of  Defense,  Manpower  Reserve  Affairs  and 
Logistics  (MRA&L),  “Enlistment  Standards,”  effective  11  September  1980. 

The  research  program  began  in  1 982  with  an  effort  known  as  Project  A.  Project 
A  not  only  validated  the  ASVAB  against  job  performance,  it  further  linked  indicators  of 
temperament  (achievement,  discipline,  stress  tolerance),  psychomotor  ability  (e.g.,  eye- 
hand  coordination),  and  spatial  ability  to  job  performance.  Project  A  developed  new 
tools  for  a  variety  of  personnel  decisions.  Before  these  tools  can  be  optimally  used, 
however,  two  critical  questions  need  to  be  answered:  (1 )  What  combinations  of 
aptitude,  temperament,  psychomotor  ability,  and  spatial  ability,  measured  at  or  before 
entry  into  the  Army,  best  predict  later  performance  in  individual  military  occupational 
specialties  (MOS)?  and  (2)  Which  indicators  of  first-tour  performance  best  predict 
performance  in  the  second  tour?  These  questions  will  be  answered  in  Building  the 
Career  Force. 

The  fourth-year  Building  the  Career  Force  activities  described  herein  continued 
analyses  focused  on  the  combined  set  of  initial  entry  predictor  measures  developed  for 
selection  and  classification  purposes,  and  end-of-training  and  first-tour  job  performance 
measures  to  be  linked  to  these  predictor  measures.  Data  from  second-tour  measures 
administered  to  a  sample  already  tested  on  initial  entry,  end-of-training,  and  first-tour 
measures  are  being  examined  for  longitudinal  linkages  across  the  full  set  of  measures 
from  initial  entry  into  second  tour.  The  information  obtained  from  these  analyses  will 
provide  a  base  for  setting  selection,  classification,  reenlistment,  and  promotion  policies 
unrivaled  anywhere. 

Active  sponsorship  of  this  effort  has  been  provided  by  the  Director  of  Military 
Personnel  Management  (DMPM).  The  DMPM  has  been  periodically  briefed  on  the 
activities  described  herein  and  has  personally  taken  part  in  the  execution  of  this  project 
in  an  extremely  effective  manner.  To  ensure  that  Building  the  Career  Force  research 


iii 


achieves  its  full  scientific  potential,  an  advisory  group  composed  of  experts  in 
personnel  measurement,  selection,  and  classification  has  provided  continuing  guidance 
on  technical  aspects  of  the  research.  Members  of  this  scientific  advisory  group  include 
Drs.  Philip  Bobko,  Lloyd  Bond,  Milton  Hakel  (Chair),  Lloyd  Humphreys.  Lawrence 
Johnson,  Robert  Linn,  Mary  Tenopyr,  and  Jay  Uhlaner. 


ZITA  M.  SIMUTIS  EDGAR  M.  JOHNSON 

Deputy  Director  Director 

(Science  and  Technology) 
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EDITORS'  PREFACE 


This  is  the  fourth  annual  report  for  work  completed  as  part  of  the 
Building  the  Career  Force  Project.  It  also  constitutes  the  primary  technical 
reoort  of  the  work  completed  on  several  of  the  project's  principal  ^^^ks. 
CoSsequeltly  it  is  a  "stand  alone"  document  for  Fisca  Year  1993  and  does  not 
refer  thfre^der  to  more  detailed  descriptions  in  supplementary  reports  for 
that  period.  The  Career  Force  Project  extends  the  major  work  on  selection  and 
classification  of  Army  enlisted  personnel  that  was  completed  as  part  of 

Project  A. 

The  Career  Force  Project  includes  (1)  a  replication  and  extension  of  the 
Fxoerimental  Predictor  Battery  validities  for  the  selection  and  classification 
enlisted  personal;  (2)  validation  of  the  Experimental  Battery 
aoainst  end-of-training  performance;  (3)  validation  of  training  performance 
a^oredictor  of  first-tour  job  performance:  (4)  measurement  of  second-tour 
oerformance-  (5)  validation  of  the  Armed  Services  Vocational  Aptitude  Battery 
(ASVAB),  the  Lperimental  Battery,  Advanced  Individual  Training  (AIT) 
oerformance  and  first-tour  performance  as  predictors  of  second-tour 
performance!  and  (6)  identification  of  the  optimal  predictor  ^^Ttery  for 
selection  and  classification,  given  certain  specific  sets  of  g 
constraints. 

The  annual  report  for  year  one  described  the  results  of  a  series  of 
analvses  directed  at  basic  score  development  for  (1)  the  Experimental 
Predictor  Battery  (2)  the  End-of-Training  performance  measures,  and  (3)  the 
econ5l?our  ^rjerfoLnce  measures  that  were  administered  to  t  a  second-tour 
Concurrent  Validation  sample  (CVII).  The  Pstformance 
sample  of  second-tour  junior  noncommissioned  officers  (NCO) 
develop  a  latent  structure  model  of  second-tour  performance.  The  model 
hypothesizes  six  basic  components  for  NCO  performance. 

The  annual  report  for  year  two  dealt  with  the  analysis  of  performance 
data  from  the  Longitudinal  Validation  I  (LVI)  sample,  which  is  a  sample  of 
aoDroximately  10  000  first-tour  incumbents  who  entered  the  Army  during 
?986/8r  It  is  the  second  of  the  two  major  cohorts  of  enlisted  personnel  that 
make  uo  the  total  Project  A/Career  Force  Project  data  base.  The  criterion 
score  development,  data  editing,  and  performance  modeling  'Jf Q^J^e 

described  in  turn.  The  remainder  of  the  report  described  the  results  of  the 
ba^-c  Longitudinal  Validation  of  the  ASVAB  and  the  Project  A  Experimental 
Predictor^Battery  against  (1)  training  performance,  (2)  Tirst-tour  job 
performance,  and  (3)  second-tour  job  performance  (i.e.,  the  second-tour 
performance  factor  scores  developed  during  year  one). 

The  third  annual  report,  for  FY92,  focused  exclusively  on  the 
Longitudinal  Validation  Second-Tour  (LVII)  sample.  This  is  a  sample  of 
approximately  1,500  individuals  in  the  nine  "Batch  MOS  who  had  ^eenlis 
and  were  2-3  years  into  their  second  tour  of  duty  at  the  time  the 
measures  of  sLond-tour  job  performance  were  administered.  The  individuals  in 
the  sample  had  entered  the  Army  in  1986/87  as  part  of  ^ 

Longitudinal  Validation  (LV)  sample.  The  report 

rnllprtion  orocedure  (2  the  editing  of  the  data  file,  (3)  the  ^oitiai 

analyses  of  each  instrument  to  develop  the  basic  criterion 

development  of  a  model  of  second-tour  performance  based  on  the  LVII  sample 


V 


data  A  Mjor  feature  of  the  results  is  the  great  consistency  in  the 
covariance  structure  of  the  basic  criterion  scores  across  cohorts  (CVII  vs. 
LVII)  and  across  organizational  levels  (LVI  vs.  LVllj. 

This  fourth  annual  report,  for  FY93,  covers  (1)  a  summary  of  the 
;,nalvses  done  with  the  Experimental  Battery  to  support  the  work  of  the 
filler  Accession  Policy  Working  Group  as  it 

+  (?)  the  basic  validation  analyses  of  ASVAB  and  Able  againsi  secunu 

™r  e?Ll  efin  ^longitudinal  sample  (LVII),  using  the  LV  I  performance 
^deMaao^r^  c?Uer,on  Scores;  (3)  the  degree  Jo  “  ^ 

performance  with  performance  exhibit  convergent  and 
nrnanizational  levels  (e.g.,  first-tour  performance  vs.  second-tour 
nerformance)  relative  to  the  individual  components  of  performance;  (4)  the 
fesSlts  of  modeling  the  predictors  of  first-tour  attrition  using  event-history 
analysis;  and  (5)  a  description  of  results  obtained  with  the  Army  Job 
Satisfaction  Questionnaire. 

The  remaining  activites  for  the  project  are  to  (1)  identify  the 
"oDtimal"  prediction  equations  that  maximize  overall  selection  validity  for 
the  Army  personnel  system,  given  constraints;  (2)  estimate  the  potential 
Sllfe^ntial  p?ediction/ciaIsification  validity  of  the  Expecimehtal  Sattery; 
(3)  analyze  the  relative  predictability  of  alternative  selection  and 
classifiSion  go^s:  and  (4)  conduct  a  complete  fairness  analysis  of  al 
maior  oredictor/cri terion  relationships  in  the  Project  A/Career  Force  data 
Sase'  ^^he  ^esult^o?  these  analyses  will  be  the  topics  of  subsequent  reports. 

As  was  the  case  for  previous  years,  the  writing  of  this  report  was  very 
much  a  collaborative  effort  by  a  lot  of  people.  The  primary  authors  for  each 
chanter  are  indicated  in  the  Contents  and  also  on  the  f^rst  page  of  each  ^ 
chapter.  The  editors,  and  the  management,  are  deeply  appreciative  of  their 

contributions. 
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BUILDING  AND  RETAINING  THE  CAREW  fOR^E:  NEW  PROCEDURES  ™  “MESSING  AND 
assigning  army  enlisted  personnel  --  ANNUAL  REPORT,  1993  FISCAL  YhAK 


Rpgui'rements: 

The  Career  Force  Project  is  second  phase  of  a 
term  research  program,  sponsored  by  the  Deputy  Chief  of  Staff  for  Personne  , 
to  improve  the^election  and  assignment  of  Army  enlisted  ^  ^ 

first^Dhase  Project  A,  existing  selection  measures  were  validated  against 
both  existing  and  newly  developed  performance  criteria,  and  new  predictive 
mpasures  were  developed  to  aid  in  assignment  and  promotion  decisions.  The 
Career  Force  project  extends  the  research  to  measure  second-tour  Joj  perform¬ 
ance  and  to  examine  how  selection  and  classification  tests  administered 
before  a  soldier's  enlistment  can,  with  measures  of  performance  during  that 
enlistment,  predict  performance  potential  for  second-tour  duty. 


Procedure: 

In  Task  1,  measures  adopted  in  Project  A  to  assess  the  performance  of 
second-tour  soldiers  were  administered  to  the  Longitudinal 
<;amDle  first  tested  in  Project  A  at  entry  and  during  their  first  tour.  I  ne 
results  from  these  tests  are  being  analyzed  to  complete  the  P'^edictive 
validation  of  the  Armed  Services  Vocational  Aptitude  Battery  (ASVAB)  and  the 
^roiecJ  A  ExpeH^ntal  Predictor  Battery,  measures  of  training  success,  and 
first-tour  job  performance  tests  against  the  criteria  of  successful  second 
^iuroe^formance  T^k  2  staff  has  established  an  integrated  data  base  and  is 
nrnrp<;sinn  Proiect  A  and  Career  Force  data  and  merging  files  with  related 
Si^itflrv  Lta  ^  Task  3  covers  all  analyses  performed  to  develop  the  analytic 
fllSk  nLded  to  evaluate  equations  for  predicting  training  performance, 
first-tour  performance  and  attrition,  reenl istment,  and  second-tour 
performance. 


Findings: 

The  oattern  of  results  from  confirmatory  analyses  of  the  latest 
Longitudinal  Validation  tests  has  continued  to  be  consi stent  with  The  results 
from  earlier  LV  testing,  as  well  as  from  the  initial  Concurrent  Validation 
tests  Tile  description  of  the  latent  structure  of  performance  as  individuals 
move  from  training^through  their  first  tour  and  into  their  second  tour 
continued  to  be  highly  consistent  as  alternative  ways  of  predicting  and 
assessing  development  and  leadership  qualities  are  tested. 

Performance  in  training  does  predict  performance  on  the  job  (first 
tour),  and  entry-level  performance  does  predict  second-tour  NCO  performance. 


The  vail 
soldiers  in  the 


iditv  of  the  ASVAB  is  as  high  for  predicting  performance  by 
leir  second  tour  as  it  was  for  predicting  first-tour  performance. 
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Its  highest  validities  are  for  predicting  Core  Technical  Proficiency  but  it 
also  hL  high  validity  for  predicting  second-term  leadership  performance. 

Components  of  the  Predictor  Test  Battery  and  measures  of  current 
performance  each  contribute  considerable  unique  variance  to  predicting  future 
performance. 

Extensive  analyses  comparing  various  combinations  of  Experimental 
Battery  tests  and  ASVAB  subtests  show  that  all  relevant  indices  (e.g., 
absolute  validity,  discriminant  validity,  subgroup  differences)  cannot  be 
maximized  by  the  same  test  battery.  In  choosing  bests  to  J 

battery,  trade-offs  will  have  to  be  made  in  terms  of  the  objectives  that  are 

considered  most  important. 

A  model  of  the  relationship  of  the  ASVAB  and  Experimental  Battery 
predictors  with  attrition  shows  that,  as  has  long  been  recognized,  high  school 
diploma  graduate  status  is  the  best  predictor  of  low  attrition  among  soldiers 
in  their  first  tour.  For  incremental  predictive  gam,  the  best  fitting 
predictor  composite  in  these  analyses  consisted  of  five  scales  i^.g. , 
Nondelinquency,  Physical  Condition)  from  the  Assessment  of  Background  and  Life 
Experiences  (ABLE)  and  the  Quantitative  subtest  composite  from  the  ASVAB. 


Utilization  of  Findings: 

The  data  from  the  completed  predictor  and  performance  analyses  from 
Proiect  A/Career  Force  are  providing  the  base  for  a  series  of  maximizing/ 
optimizing  analyses  of  selection  and  classification  efficiency.  These  results 
will  be  integrated  with  previous  validation  and  modeling  work  to  develop 
comprehensive  procedures  for  estimating  potential  gains  to  the  Army  yoni 
classification  changes  under  varied  psychometric,  organizational,  and  labor 
market  conditions. 
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BUILDING  AND  RETAINING  THE  CAREER  FORCE:  NEW  PROCEDURES  FOR  ACCESSING  AND 
ASSIGNING  ARMY  ENLISTED  PERSONNEL  --  ANNUAL  REPORT,  1993  FISCAL  YEAR 


Chapter  1 
INTRODUCTION 

John  P.  Campbell  and  James  H.  Harris 


This  report  is  a  summary  of  the  major  activities  undertaken  during  the 
fourth  year  of  a  Department  of  the  Army  research  project  entitled  Building  and 
Retaining  the  Career  Force.  The  report  covers  the  period  of  the  1993  fiscal 
year,  beginning  1  October  1992.  The  research  reported  was  conducted  by  a 
consortium  comprised  of  Human  Resources  Research  Organization  (HumRRO), 
American  Institutes  for  Research  (AIR),  and  Personnel  Decisions  Research 
Institute,  Incorporated  (PDRI,  Inc.),  under  contract  to  and  in  collaboration 
with  the  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences 
(ARI). 


The  research  effort  is  the  second  phase  of  a  two-phase  program  to 
develop  a  selection  and  classification  system  for  enlisted  personnel,  based  on 
expected  future  performance.  Phase  One  was  Project  A  (Campbell  &  Zook,  1991). 
Its  goals  were  to  validate  the  Armed  Services  Vocational  Aptitude  Battery 
(ASVAB)  by  collecting  data  from  a  representative  sample  of  Military 
Occupational  Specialties  (MOS),  and  to  build  a  large  and  versatile  data  base 
by  developing  and  validating  new  predictors  and  criterion  measures  that 
represented  the  entire  domain  of  potential  measures. 

The  goals  of  Building  the  Career  Force  are  to  determine  the 
longitudinal  relationship  between  the  new  predictors  and  first-tour 
performance,  to  finalize  and  administer  the  measures  of  second-tour  job 
performance,  and  to  examine  how  selection  and  classification  tests 
administered  before  a  soldier's  first  enlistment,  in  conjunction  with 
performance  during  that  soldier's  first  enlistment,  predict  performance  in  a 
second  enlistment. 

The  remainder  of  this  chapter  describes  the  objectives  and  organization 
of  the  project,  summarizes  the  work  completed  during  the  first  years  of  the 
Project,  and  outlines  the  content  included  in  this  fourth  annual  report. 


BUILDING  THE  CAREER  FORCE:  OBJECTIVES  AND  PROJECT  DESIGN 

The  Project  A  data  base^  the  predictor  and  criterion  measures  the 
project  developed,  the  working  models  it  provided,  and  its  basic  analytic  work 
have  provided  a  valuable  foundation  for  the  further  production  of  scientific 
findings  and  operational  products,  and  for  the  subsequent  investigation  of 
reenlistment  decisions,  noncommissioned  officer  (NCO)  job  performance,  NCO 
promotion  decisions,  and  the  identification  of  NCO  potential. 

The  work  encompassed  by  the  Career  Force  Project  is  intended  to 
accomplish  several  general  goals  relevant  to  building  and  retaining  the  career 
force.  The  goals  may  be  summarized  as  follows. 
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(1)  Build  the  final  pieces  required  for  a  complete  selection/ 
classification  decision-making  system  for  Army  enlisted  personnel. 

(2)  Provide  the  analytic  procedures  and  data  necessary  to  maximize  the 
system's  performance  and  evaluate  its  effectiveness. 

(3)  Build  the  foundation  for  its  implementation. 

The  principal  focus  is  on  the  greatest  possible  gains  in  overall 
individual  performance,  for  both  "can  do"  and  "will  do"  components  of 
performance,  that  can  be  obtained  from  enhancing  the  selection/classification 
system  for  first-  and  second-tour  enlisted  personnel.  Maximizing  the  benefit 
from  a  more  effective  match  of  people  and  jobs  has  always  been  a  goal  of  the 
Army  personnel  system.  Given  the  population  demographics  in  prospect  for  the 
United  States  during  the  current  decade,  this  goal  becomes  even  more  crucial. 
It  is  incumbent  on  virtually  every  organization  to  go  as  far  as  the  state-of- 
the-art  will  al low. 

This  means  that  the  information  used  to  make  personnel  decisions  must 
yield  the  maximum  gain  in  terms  of  accuracy  and  fairness  of  predictions.  It 
means  that  the  models  and  procedures  used  to  execute  selection  and 
classification  decisions  (e.g.,  new  computerized  tests)  must  both  serve  the 
goals  of  the  organization  and  maximize  the  aggregate  benefits  that  can  be 
obtained  from  using  the  selection/  classification  measures.  It  means  that  the 
implementation  of  the  system,  or  any  part  of  it,  must  serve  the  needs  of  the 
users  and  also  maintain  fidelity  with  the  goals  on  which  the  system  is  based. 

Specific  Research  Objectives 

The  specific  scientific  objectives  of  Building  the  Career  Force  are  to: 

(1)  Develop  a  complete  array  of  valid  and  reliable  measures  of  second- 
tour  performance  as  an  Army  NCO,  using  the  Project  A  prototypes  as 
a  starting  point. 

(2)  Carry  out  a  complete  incremental  predictive  validation  of  (a)  the 
ASVAB  and  the  Project  A  Experimental  Battery  of  predictors,  (b) 
measures  of  training  success,  and  (c)  the  full  array  of  first-tour 
performance  criteria  developed  as  part  of  Projec^t  A.  The  criteria 
against  which  these  three  sets  of  predictors  will  be  validated, 
both  individually  and  incrementally  for  each  major  criterion 
component,  are  the  second-tour  job  performance  measures. 

(3)  Develop  a  model  of  second-tour  soldier  performance  that  parallels 
the  first-tour  performance  model  from  Project  A  and  that 
identifies  the  major  components  of  second-tour  performance, 
provides  information  on  their  construct  validity,  and  establishes 
how  the  components  should  be  combined  for  specific  prediction  or 
interpretation  purposes. 

(4)  Develop  the  analytic  framework  needed  to  evaluate  the  optimal 
prediction  equations  for  predicting  (a)  training  performance; 

(b)  first-tour  performance;  (c)  first-tour  attrition  and  the 
reenlistment  decision;  and  (d)  second-tour  performance,  under  the 
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conditions  when  testing  time  is  limited  to  a  specified  amount  and 
when  there  must  be  a  tradeoff  among  alternative  selection/ 
classification  goals  (e.g.,  maximizing  aggregate  performance  vs. 
minimizing  discipline  and  low-motivation  problems  vs.  minimizing 
attrition) . 

(5)  Design  and  develop  a  fully  functional  and  user-friendly  research 
data  base  that  includes  all  relevant  personnel  data  on  1981/82, 
1983/84,  and  1986/87  accessions,  including  all  Project  A  and 
Career  Force  Project  data  and  all  relevant  data  from  the  Enlisted 
Master  File  (EMF),  Accession  File,  and  Army  Training  Requirements 
and  Resources  System  (ATRRS). 

Project  Organization 

To  reflect  the  requirements  of  the  Career  Force  research,  the  project 
is  organized  as  shown  in  Figure  1.1.  Management  of  the  total  project  is  the 
responsibility  of  the  Project  Director.  The  overall  design,  execution,  and 
evaluation  of  the  substantive  tasks  are  the  responsibility  of  the  Principal 
Scientist.  Oversight  and  scientific  participation  are  provided  by  the  Army 
Research  Institute.  Guidance  has  been  provided  by  the  General  Officers 
Steering  Committee  and  the  Scientific  Advisory  Group. 

A  brief  summary  of  the  work  encompassed  by  the  three  substantive 
technical  tasks  follows: 

Task  1  deals  with  revision  of  the  measures  developed  in  Project  A  to 
measure  second-tour  soldier  performance.  The  second-tour  performance  measures 
were  revised  and  were  administered  to  the  Project  A  Longitudinal  Validation 
(LV)  sample,  beginning  in  June  1991.  At  that  time,  the  soldiers  in  the  sample 
were  in  their  second  tour,  and  had  been  in  the  Army  anywhere  from  41  to  63 
months.  Once  the  data  have  been  fully  analyzed  (under  Task  3),  it  will  be 
possible  to  complete  the  incremental  predictive  validation  of  the  ASVAB  and 
the  Project  A  Experimental  Battery,  the  measures  of  training  success,  and  the 
full  array  of  first-tour  performance  measures  developed  in  Project  A,  against 
the  second-tour  criterion  measures. 

Task  2  has  a  single  purpose--to  establish,  manage,  and  safeguard  an 
integrated  research  data  base  (IRDB).  As  part  of  the  establishment  of  the 
IRDB,  Task  2  is  integrating  the  Project  A  longitudinal  research  data  base, 
extracting  and  merging  data  from  other  military  data  bases,  processing  data 
collected  by  Project  A  and  this  project,  and  creating  workfiles  for  analyses. 

Task  3  is  responsible  for  all  analyses  performed  under  this  project. 

The  task  is  organized  around  the  five  major  data  sets  to  be  analyzed:  the 
Longitudinal  Validation  predictor  data  (LV),  the  Longitudinal  Validation  end- 
of-training  (EOT)  data,  the  Longitudinal  Validation  first-tour  data  (LVU,  the 
Concurrent  Validation  second-tour  data  (CVII),  and  the  Longitudinal  Validation 
second-tour  data  (LVII).  At  the  end  of  the  project.  Task  3  will  have 
developed  the  analytic  framework  necessary  to  evaluate  optimal  prediction 
equations  to  predict  training  performance,  first-tour  performance  and 
attrition,  reenlistment,  and  second-tour  performance. 
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Figure  1 .1 .  Building  the  Career  Force:  Project  management  structure. 


Project  Design 


As  will  be  explained  in  later  sections  of  this  chapter,  the  remaining 
chapters  of  this  report  all  deal  with  the  analyses  of  data  obtained  at  one 
major  point  in  the  total  project  design.  To  set  the  stage  for  these 
discussions,  as  well  as  for  the  summary  of  work  done  during  years  one  through 
three,  the  basic  overall  project  design  is  summarized  below. 

The  Research  Sample 

In  general,  the  combined  design  for  Project  A/Career  Force  encompasses 
two  major  cohorts  of  soldiers  (new  accessions  for  1983/84  and  for  1986/87), 
each  of  which  were  followed  into  their  second  tour  of  duty  and  which  collect- 
ively  have  produced  six  major  research  samples.  For  each  research  sample 
there  is  a  battery  of  predictor  measures  and  an  array  of  performance  measures. 
For  each  of  the  six  samples  the  predictor  battery  is  composed  of  the  ASVAB  and 
either  the  Trial  Battery  or  the  Experimental  Battery  version  of  the  new  tests 
developed  in  Project  A  (see  Campbell  &  Zook,  1991).  There  were  three  distinct 
arrays  of  performance  measures  corresponding  to  the  need  to  assess  (a) 
training  performance,  (b)  first-tour  job  performance,  and  (c)  second-tour  job 
performance. 

In  each  sample  the  individuals  to  be  assessed  were  selected  from  two 
predetermined  sets  of  MOS  —  Batch  A  and  Batch  Z.  They  are  listed  in  Figure 
1.2.  The  two  groups  differed  in  that  tests  administered  to  Batch  A  MOS 
included  MOS-specific  rating  scales  and  job  knowledge  and  hands-on  tests, 
whereas  the  only  MOS-specific  measure  administered  to  the  Batch  Z  MOS  was  an 
end-of-training  test. 


Batch  A 

Batch  Z 

_ _ _ — — - - - - - - - - - 

MOS 

MOS 

11B 

13B 

19E 

19K 

31C 

63B 

71L 

88M 

91A/B 

95B 

Infantryman 

Cannon  Crewmember 

M60  Armor  Crewman 

M1  Armor  Crewman® 

Single  Channel  Radio  Operator 
Light-Wheel  Vehicle  Mechanic 
Administrative  Specialist 

Motor  Transport  Operator^ 

Medical  Specialist/Medical  NCO*^ 
Military  Police 

12B 

16S 

27E 

29E 

51B 

54B 

55B 

67N 

76Y 

94B 

96B 

Combat  Engineer 

MANPADS  Crewman 

Tow/Dragon  Repairer 

Comm.-Electronics  Radio  Repairer 
Carpentry/Masonry  Specialist 

NBC  Specialist*' 

Ammunition  Specialist 

Utility  Helicopter  Repairer 

Unit  Supply  Specialist 

Food  Service  Specialist 

Intelligence  Analyst 

®  Except  1 
This  MC 
^  Ahhoug^ 
redassT 
1  ^  This  MC 

tor  the  type  of  tank  used,  this  MOS  is  equivalent  to  the  19E  MOS  originally  selected  for  Project  A  testing. 

IS  was  formerly  designated  as  64C. 

1  91A  was  the  MOS  originally  selected  for  Project  A  testing,  second-tour  medical  specialists  are  usually 
Redas91B, 

)S  was  formerly  designated  as  54E. 

Figure  1.2.  Project  A/Career  Force  Military  Occupational  Specialties  (MOS) 
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The  MOS  in  the  two  groups  were  carefully  sampled  to  represent  the 
variation  in  job  content  in  the  Army  occupational  structure.  In  addition, 
they  were  selected  so  as  to  overrepresent  both  the  combat  specialties  and 
those  MOS  with  the  larger  proportions  of  women  and  minority  groups.  The  MOS 
selection  procedure  has  been  described  in  detail  in  previous  Project  A  reports 
(e.g.,  Campbell,  1987). 

A  glossary  of  terms  for  the  samples  and  for  the  different  measurement 
batteries  is  given  in  Figure  1.3.  The  six  major  samples,  their  approximate 
size,  and  the  predictor  and/or  performance  batteries  that  were  to  be 
administered  to  each  are  shown  in  Figure  1.4. 


Glossary  of  Terms 

CVI  Sample  (CVI) 

Soldiers  who  entered  the  Army  between  1  Jul  83  -  30  Jun  84  and  were  in  the 

1985  Project  A  Concurrent  Validation  (CVI).  They  were  administered  the  Trial 
Predictor  Battery  and  the  first-tour  job  performance  measures. 

evil  Sample  (CVII) 

Soldiers  who  entered  the  Army  between  1  Jul  83  -  30  Jun  84  and  were  in  the 

1985  Project  A  Concurrent  Validation  (CVI)  and  the  1988  Second-Tour 
Concurrent.  Validation  (CVII).  They  were  administered  the  second-tour  job 
performance  measures  and  were  re-administered  the  ABLE  assessment 
instrument. 

LV  Sample  (LV) 

Soldiers  in  the  Longitudinal  Validation  sample  who  entered  the  Army  between 

20  Aug  86  -  30  Nov  87  and  were  administered  the  Experimental  Predictor 

Battery  and  End-of-Training  measures. 

LV  Training 

Sample  (LVT) 

Soldiers  in  the  Longitudinal  Validation  sample  who  finished  Advanced 

Individual  Training  (AIT)  and  who  were  administered  the  End-of-Training 
measures. 

LVI  Sample  (LVI) 

Soldiers  who  entered  the  Army  between  20  Aug  86  -  30  Nov  87  and  were  in  the 
LV  Sample  and  the  1988  First-Tour  Longitudinal  Validation  sample  (LVI). 

They  were  administered  the  first-tour  job  performance  measures. 

LVII  Sample  (LVII) 

Soldiers  who  entered  the  Army  between  20  Aug  86  -  30  Nov  87  and  were  in  the 
LV  Sample  and  the  LVI  Sample  and  the  Longitudinal  Validation  Second-Tour 
(LVII)  sample.  They  were  administered  the  second-tour  job  performance 
measures  in  LVII. 

Note.  Glossary  definitions  reflect  the  original  research  plan.  In  the  course  of  the  research,  some  CVII 
soldiers  did  not  have  CVI  data,  some  LVI  soldiers  did  not  have  LV  data,  and  some  LVII  soldiers 
did  not  have  both  LV  and  LVI  data. 

Figure  1.3.  Glossary  of  Terms  for  Project  A/Career  Force  Research  Samples 


Procedure 


The  data  collection  procedures  for  each  sample  have  been  described  in 
detail  in  previous  annual  reports  (e.g.,  see  Campbell  &  Zook,  1990).  Each 
data  collection  involved  on-site  administration  by  a  trained  data  collection 
team  headed  by  a  team  leader  from  the  contractor  staff  who  worked  closely  with 
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RIe 

Data 

Analyses 


Predictor 
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Performance 
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N  =  9,500 
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N  =  1,000 


86/87  Longitudinal  Sample 


Experimental  End-of- 
Predictor  Training 

Battery  Performance 

Measures 

N  =  45,000 

N  =  30,000 
LV 

Lvr 


First’Tour 
Performance 
Measures 
N  =  10,000 
LVI 


86/87 

Longitudinal 

Sample 


Second’Tour 
Performance 
Measures 
N=  1.500 
LVll 


Figure  1.4.  Career  Force  Research  Flow  and  Samples 

a  designated  Army  point-of-contact  (POC)  at  the  site.  A  brief  characteriza¬ 
tion  of  each  of  the  six  samples  in  terms  of  the  timing,  location,  and  duration 
(per  soldier)  of  the  data  collection  follows. 

The  Concurrent  Validation  First-Tour  (CVI)  sample.  The  data  were 
collected  at  13  posts  in  the  continental  United  States  and  at  multiple 
locations  in  Germany.  Each  individual  was  assessed  for  1  1/2  days  on  the 
project-developed  first-tour  job  performance  measures  and  for  1/2  day  on  the 
new  predictor  measures  (the  Trial  Battery).  The  individuals  in  the  sample  had 
been  in  the  Army  for  12-24  months.  Data  analysis  has  been  completed  for  this 
sample. 

The  Longitudinal  Validation  Predictor  (LV)  Sample.  All  individuals  were 
assessed  on  the  4-hour  Experimental  Predictor  Battery  within  2  days  of  first 
arriving  at  their  assigned  Reception  Battalion  where  they  would  undergo 
Basic/Advanced  Individual  training.  Data  were  collected  over  a  14-month 
period  at  eight  Reception  Battalions  by  a  permanent,  on-site  data  collection 
team. 
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The  Longitudinal  Validation  End-of-Traininq  (LVT)  Sample.  The  EOT 
performance  measures  were  administered  to  those  individuals  in  the  LV  sample 
who  completed  Advanced  Individual  Training  (AIT),  which  could  take  from  2 
months  to  6  months,  depending  on  the  MOS.  The  training  performance  measures 
consisted  of  an  MOS-specific  training  achievement  test  and  a  series  of  rating 
scales  completed  by  peers  and  drill  instructors.  Data  collection  took  place 
during  the  last  three  days  of  AIT. 

The  Longitudinal  Validation  First-Tour  (LVI)  Sample.  The  individuals  in 
the  86/87  cohort  who  were  measured  with  the  Experimental  Predictor  Battery, 
completed  AIT,  and  remained  in  the  Army  were  assessed  with  the  full  array  of 
first-tour  job  performance  measures  when  they  were  between  12  and  24  months  of 
service.  Data  collections  were  conducted  at  13  posts  in  the  United  States  and 
multiple  locations  in  Europe  (primarily  in  Germany).  The  administration  of 
the  LVI  first-tour  criterion  measures  took  one  day  per  soldier. 

The  Concurrent  Validation  Second-Tour  (CVII)  Sample.  The  same  data 
collection  teams  that  administered  the  first-tour  performance  measures  to  the 
LVI  sample  also  administered  the  second-tour  performance  measures  at  the  same 
location  and  during  the  same  time  periods  to  a  sample  of  junior  NCOs  from  the 
83/84  cohort  in  their  second  tour  of  duty  (3-5  years  of  service).  Every 
attempt  was  made  to  include  second-tour  personnel  from  the  designated  MOS  who 
had  been  part  of  the  first-tour  Concurrent  Validation  sample  (CVI).  The  CVII 
data  collection  took  one  day  per  soldier. 

The  Longitudinal  Validation  Second-Tour  (LVII)  Sample.  The  personnel  in 
this  sample  are  members  of  the  86/87  cohort  from  the  designated  MOS  who  were 
part  of  the  LV  (predictors  and  training  performance  measures)  and  LVI  (first- 
tour  job  performance  measures)  samples  and  who  reenlisted  for  a  second  tour  of 
duty.  The  revised  second-tour  performance  measures  were  administered  at  15 
U.S.  posts,  multiple  locations  in  Germany,  and  two  locations  in  Korea.  The 
LVII  performance  assessment  took  one  day  per  soldier. 


SUMMARY  OF  PROJECT  EFFORTS  FOR  YEAR  ONE 

As  described  in  the  first  Career  Force  annual  report  (Campbell  &  Zook, 
1990),  the  objectives  of  the  project's  first  year  were  focused  on  developing  a 
full  design  for  the  data  base  and  on  analyzing  basic  scores  for  (a)  the  final 
version  of  the  Experimental  Predictor  Battery  (EB),  (b)  the  End-of-Training 
(EOT)  performance  measures,  and  (c)  the  second-tour  criterion  measures  used  to 
assess  NCO  performance  in  the  second-tour  Concurrent  Validation  (CVII)  sample. 
The  data  from  the  End-of-Training  and  second-tour  Concurrent  Validation  (CVII) 
performance  assessments  were  also  used  to  formulate  both  a  model  of  training 
performance  and  a  model  of  second-tour  (junior  NCO)  job  performance.  That  is, 
the  basic  scores  from  the  individual  performance  measures  were  aggregated  into 
factor  scores  that  represented,  as  well  as  possible,  the  major  components,  or 
latent  structure,  of  training  performance  and  second-tour  job  performance. 

By  the  end  of  year  one,  the  data  collection  for  the  Longitudinal 
Validation  first-tour  performance  assessments  had  been  completed,  but  the  data 
cleaning  and  editing  were  still  in  progress  and  the  analysis  of  the  LVI 
performance  measures  had  not  yet  begun. 
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Data  Base  Design 


As  described  in  the  first-year  annual  report,  the  Career  Force  data  base 
design  allows  access  at  any  level  of  score  aggregation.  The  report  describes 
each  variable  and  the  amount  of  information  that  is  available.  The  data  are 
accessed  via  a  secure  system  that  requires  prior  approval  by  the  Army.  The 
data  base  also  includes  data  from  several  operational  files  maintained  by  the 
Army. 


Basic  Scores  for  the  Experimental  Battery 

During  year  one,  much  effort  was  devoted  to  analyzing  the  data  that  had 
been  obtained  by  administering  the  Experimental  Predictor  Battery  to  almost 
50,000  new  accessions  in  the  Longitudinal  Validation  sample.  A  number  of  data 
editing  procedures  were  compared  and  evaluated,  and  great  care  was  taken  to 
maximize  data  quality  for  the  information  that  was  entered  into  the  final  data 
file.  The  psychometric  properties  and  subgroup  differences  for  each  measure 
were  analyzed,  and  a  series  of  exploratory  and  confirmatory  analyses  were 
conducted  to  identify  the  basic  predictor  scores  within  each  domain  that  would 
be  used  in  the  validation  analyses. 

The  final  array  of  tests  in  the  Experimental  Battery  and  the  constructs 
they  are  intended  to  measure  are  shown  in  Figure  1.5.  The  31  basic  scores 
that  are  obtained  from  the  specific  test  indicators  are  shown  in  Figure  1.6 
(Campbell  &  Zook,  1990). 

There  was  a  very  high  degree  of  consistency  between  the  Concurrent 
Validation  and  the  Longitudinal  Validation  in  terms  of  the  factor  structures 
of  the  various  measures.  The  resulting  definitions  of  the  basic  predictor 
scores  to  be  used  in  the  validation  analyses  were  quite  similar. 

Basic  Scores  for  the  End-of -Training  Measures 

During  year  one,  the  data  from  the  school  knowledge  test  and  seven 
training  performance  rating  scales  administered  at  the  end  of  training  were 
analyzed  in  terms  of  their  psychometric  properties  and  factor  structure. 
Confirmatory  techniques  were  used  to  identify  the  "model"  of  training 
performance  that  best  represented  the  covariances  among  the  observed  measures. 
That  is,  an  a  priori  set  of  alternative  models  was  proposed  and  evaluated  in 
terms  of  the  degree  to  which  they  fit  the  data.  In  the  end  six  basic  scores 
were  proposed,  two  based  on  the  knowledge  tests  and  four  based  on  the  rating 
scales.  A  brief  characterization  of  the  six  scores  is  given  in  Figure  1.7 
(Campbell  &  Zook,  1990). 

These  six  scores  will  serve  both  as  criterion  measures  (for  the 
Experimental  Battery)  and  as  predictors  (of  first-tour  and  second-tour  job 
performance)  in  later  validation  analyses. 
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Test/Measure 

Construct 

Paper-and-Penci  1  Spatial  Tests 

Assembling  Objects 

Object  Rotation 

Maze 

Orientation 

Map 

Reasoning 

Spatial  Visualization-Rotation 

Spatial  Visualization-Rotation 

Spatial  Visualization-Scanning 

Spatial  Orientation 

Spatial  Orientation 

Induction 

Computer-Administered  Tests 

Simple  Reaction  Time 

Choice  Reaction  Time 

Short-Term  Memory 

Perceptual  Speed  and  Accuracy 
Target  Identification 

Target  Tracking  1 

Target  Shoot 

Target  Tracking  2 

Number  Memory 

Cannon  Shoot 

Reaction  Time  (Processing  Efficiency) 
Reaction  Time  (Processing  Efficiency) 
Short-Term  Memory 

Perceptual  Speed  and  Accuracy 

Perceptual  Speed  and  Accuracy 

Psychomotor  Precision 

Psychomotor  Precision 

Multi  1 imb  Coordination 

Number  Operations 

Movement  Judgment 

Temperament,  Interest,  and  Job  Preference  Measures 

Assessment  of  Background 
and  Life  Experiences  (ABLE) 

Adjustment 

Dependabi 1 ity 

Achievement 

Physical  Condition 

Leadership  (Potency) 

Locus  of  Control 

Agreeableness/Likabi 1 ity 

Army  Vocational  Interest 

Career  Examination  (AVOICE) 

Realistic  Interest 

Conventional  Interest 

Social  Interest 

Investigative  Interest 

Enterprising  Interest 

Artistic  Interest 

Job  Orientation  Blank  (JOB) 

Job  Security 

Serving  Others 

Autonomy 

Routine  Work 

Ambition/Achievement 

Figure  1.5.  Experimental  Predictor  Battery  Tests  and  Relevant  Constructs 
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Figure  1.6.  Longitudinal  Validation  Experimental  Battery:  Composite  Scores  and  Constituent  Basic  Scores 


EOT  RATING  SCALE  BASED  SCORES 

1) 

Effort  and  Technical  Skill  (ETS) 

Technical  Knowledge/Skill: 

How  effective  is  each  soldier  in 
acquiring  job/soldiering  knowledge 
and  skill? 

Effort: 

How  effective  is  each  soldier  in 
displaying  extra  effort? 

2) 

Maintaining  Personal  Discipline 

(MPD) 

Following  Regulations 
and  Orders: 

How  effective  is  each  soldier  in 
adhering  to  regulations,  orders,  and 

SOP  and  displaying  respect  for 
superiors? 

Self  Control: 

How  effective  is  each  soldier  in 
controlling  own  behavior  related  to 
aggressive  acts? 

3) 

Physical  Fitness  and  Military 
Bearing  (PFB) 

Military  Appearance: 

How  effective  is  each  soldier  in 
maintaining  proper  military 
appearance? 

Physical  Fitness: 

How  effective  is  each  soldier  in 
maintaining  military  standards  of 
physical  fitness? 

4) 

Leadership  Potential  (LEAD): 

Leadership  Potential: 

Evaluate  each  soldier  on  his  or  her 
DOtential  effectiveness  as  a  leader. 

Do  not  necessarily  rate  on  the  basis 

of  present  performance. 

EOT  KNOWLEDGE 

TEST  BASED  SCORES 

5) 

Basic  Knowledge  Score: 

Items  measuring  knowledge 
requirements  common  to  all  MOS. 

6) 

Technical  Knowledge  Score: 

Items  measuring  technical  knowledge 
requirements  specific  to  each  MOS. 

Figure  1.7.  Composite  Scores  That  Reflect  End-of-Training  (EOT) 
Performance  Factors 
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Development  of  Second-Tour  Performance  Scores  (CVII) 

The  performance  measures  used  in  the  CVII  sample,  and  their  development, 
have  been  described  in  detail  in  previous  Project  A  and  Career  Force  reports 
(e.g.,  Campbell,  1991;  Campbell  &  Zook,  1991).  First-tour  measures  were 
revised  for  use  with  second-tour  personnel  and  new  measures  reflecting  the 
unique  components  of  second-tour  jobs  were  added.  A  summary  description  of 
the  specific  measures  is  given  below. 

Rating  Scales 

On  the  basis  of  second-tour  critical  incident  analyses,  the  Army-wide 
Behavioral ly  Anchored  Ratings  Scales  (BARS)  and  MOS-specific  BARS  were  revised 
and  scales  having  to  do  with  leadership  and  supervision  were  added.  Further, 
based  on  job  analysis  data,  seven  new  scales  pertaining  to  supervision  and 
leadership  responsibilities  were  also  added.  A  full  list  of  the  Army-wide 
rating  scales  is  shown  below.  Not  shown  are  the  MOS  BARS  for  each  MOS,  which 
were  revised  to  reflect  second-tour  performance  demands,  and  the  Combat 
Performance  Prediction  Scales,  which  were  the  same  as  those  used  in  LVI,  and 
which  were  not  administered  to  female  NCOs. 

Army-Wide  Behavior  Scales: 

1.  Demonstrating  Technical  Knowledge  and  Skill 

2.  Demonstrating  Effort 

3.  Supervising  Subordinates 

4.  Following  Regulations  and  Orders 

5.  Demonstrating  Integrity 

6.  Training  and  Development  of  Subordinates 

7.  Maintaining  Equipment 

8.  Physical  Fitness 

9.  Self-Development 

10.  Showing  Consideration  for  Subordinates 

11.  Demonstrating  Appropriate  Military  Bearing 

12.  Demonstrating  Appropriate  Self-Control 

Additional  Leadership  Scales: 

13.  Serving  as  a  Role  Model 

14.  Communication  With  Subordinates 

15.  Personal  Counseling 

16.  Monitoring  Subordinate  Performance 

17.  Organizing  Missions/Operations 

18.  Personnel  Administration 

19.  Performance  Counseling 

General  Scales; 


20.  Overall  Effectiveness 

21.  Senior  NCO  Potential 


Situational  Judgment  Test  (SJT) 


A  new  paper-and-penci 1  measure  of  supervisory  judgment  was  developed  by 
describing  prototypical  judgment  situations  and  asking  the  respondent  to 
select  the  most  appropriate  and  the  least  appropriate  courses  of  action.  The 
situation  descriptions  and  the  scoring  keys  were  refined  through  extensive 
subject  matter  expert  (SME)  judgments. 

Supervisory  Simulation  Exercises 

These  measures  were  developed  to  assess  NCO  performance  in  job  areas 
that  were  judged  to  be  best  assessed  through  the  use  of  interactive  exercises. 
The  simulations  were  designed  to  evaluate  performance  in  counseling  and 
training  subordinates.  A  trained  evaluator  (role  player)  played  the  part  of  a 
subordinate  to  be  counseled  or  trained  and  the  examinee  assumed  the  role  of  a 
first-line  supervisor  who  was  to  conduct  the  counseling  or  training. 

Evaluators  also  scored  the  examinee's  performance,  using  a  standard  set  of 
rating  scales. 

Here  are  brief  descriptions  of  the  three  simulation  exercises: 

-  Personal  Counseling  Simulation:  A  private  first  class  (PEC)  is 
exhibiting  declining  job  performance  and  personal  appearance. 
Recently,  the  PEC's  wall  locker  was  left  unsecured.  The 
supervisor  has  decided  to  counsel  the  PFC  about  these  matters. 

-  Disciplinary  Counseling  Simulation:  There  is  convincing  evidence 
that  the  PFC  lied  to  get  out  of  coming  to  work  today.  The  PFC  has 
arrived  late  to  work  on  several  occasions  and  has  been  counseled 
for  lying  in  the  past.  The  PFC  has  been  instructed  to  report  to 
the  supervisor's  office  immediately. 

Training  Simulation;  The  commander  will  be  observing  the  unit 
practice  formation  in  30  minutes.  The  private,  although  highly 
motivated,  is  experiencing  problems  with  the  hand  salute  and  about 
face. 

For  each  exercise,  examinee  performance  was  evaluated  on  3-point  rating 
scales  reflecting  specific  behaviors  tapped  by  the  exercises  and  a  5-point 
overall  effectiveness  rating  scale.  Factor  analyses  of  the  ratings  data 
suggested  that  each  simulation  could  be  scored  in  terms  of  the  content  of  the 
NCO's  behavior  (i.e.,  did  he  or  she  do  or  say  the  right  things)  and  the 
process .  or  style,  with  which  the  counseling  steps  were  carried  out. 

Administrative  Measures 


The  self-report  Personnel  File  Form  (PFF)  used  in  LVI  was  modified  for 
use  with  second  tour  and  six  administrative  indices  of  performance  were 
obtained. 


14 


Job  Knowledge  and  Hands-On  Measures 


The  content  of  each  of  these  measures  was  revised  on  the  basis  of  the 
second-tour  job  analyses  and  the  revised  instruments  were  subjected  to 
extensive  SME  review.  Analyses  of  alternative  aggregations  of  item  and  scale 
scores  from  both  of  these  measures  resulted  in  the  adoption  of  a  general 
(Army-wide)  and  an  MOS-specific  score  for  each  of  them. 

Final  Array  of  Second-Tour  Basic  Performance  Scores 

After  extensive  analyses  of  their  psychometric  properties  and  factor 
structures,  based  on  CVII  data,  the  final  array  of  basic  second-tour 
performance  scores  was  as  shown  in  Figure  1.8.  There  were  22  basic  scores. 
Scores  from  this  array  became  the  basis  for  the  second-tour  performance 
modeling  analysis  in  CVII. 

Development  of  the  CVII  Second-Tour  Performance  Model 

The  basic  CVII  performance  scores  served  as  input  to  the  development  of 
a  latent  structure  model  for  second-tour  performance  (Campbell  &  Zook,  1990). 
Based  on  a  consensus  of  the  project  staff,  three  major  alternatives  could  be 
used  to  explain  the  observed  correlations.  Consequently,  the  competing  models 
that  were  evaluated  for  comparative  goodness  of  fit,  using  the  LISREL  VI 
program  (Joreskog  &  Sorbom,  1986),  were  the  following: 

(1)  First-Tour  Model:  Included  five  substantive  and  two  methods 
factors,  with  the  SJT  and  Simulation  variables  all  loading  on  the 
Effort  and  Leadership  factor. 

(2)  Leadership  Factor  Model:  Included  a  sixth  substantive  factor  with 
the  SJT,  Simulation,  and  Leadership  Rating  factor  variables  all 
loading  on  this  factor.  This  model  was  evaluated  with  and  without 
a  separate  simulation  "methods"  factor. 

(3)  Training  and  Counseling  Factor  Model:  Included  a  sixth 
substantive  factor  with  just  the  Simulation  variables.  No 
separate  simulation  methods  factor  could  be  estimated  under  this 
model. 

• 

Of  the  three  models,  the  Training  and  Counseling  Factor  Model  provided 
the  closest  fit  to  the  observed  data.  A  result  of  considerable  interest  was 
that  the  SJT  (a  paper-and-penci 1  measure)  fit  best  with  the  Effort  and 
Leadership  factor,  in  spite  of  the  method  variance  involved. 

The  basic  scores  that  have  been  used  to  represent  the  latent  variables 
are  as  shown  in  Figure  1.9.  For  validation  analysis  purposes,  the  six 
substantive  factor  scores  are  obtained  by  standardizing  and  summing  the  basic 
scores  within  each  factor. 
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Hands-On  Performance  Test 

1.  MOS-specific  task  performance  score 

2.  General  (common)  task  performance  score 

Job  Knowledge  Test 

3.  MOS-specific  task  knowledge  score 

4.  General  (common)  task  knowledge  score 

Army-Wide  Rating  Scales 

5.  Leadership/supervision  composite 

6.  Technical  skill  and  effort  composite 

7.  Personal  discipline  composite 

8.  Physical  fitness  and  military  bearing  composite 

HOS-Specific  Rating  Scales 

9.  Overall  MOS  composite 

Combat  Performance  Prediction  Scales 

10.  Overall  Combat  Prediction  scale  composite  (available  for  males  only) 
Personnel  File  Form 

11.  Awards  and  Certificates 

12.  Articles  15/Flag  Actions  (Disciplinary  Actions) 

13.  Physical  Readiness 

14.  M16/M19  Qualification 

15.  Military  Training  Courses 

16.  Promotion  Rate 

Situational  Judgment  Test 

17.  Total  score  obtained  by  subtracting  the  total  "ineffectiveness" 
score  from  the  total  "effectiveness"  score 

Supervisory  Simulation  Exercises 

18.  Personal  Counseling:  Process 

19.  Personal  Counseling:  Content 

20.  Disciplary  Counsel ing:  Process 

21.  Disciplary  Counseling:  Content 

22.  Training:  Total  composite  score 


Figure  1.8.  Summary  List  of  CVII  Basic  Criterion  Scores 
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Latent  Variables  in  the  CVII  Performance  Model 


•  Core  Technical  Proficiency  (CTP) 

-  MOS-Specific  Hands-On 

-  MOS-Specific  Job  Knowledge 

•  General  Soldiering  Proficiency  (GSP) 

-  General  (Common)  Hands-On 

-  General  (Common)  Job  Knowledge 

•  Effort  and  Leadership  (ELS) 

-  Awards  and  Certificates 

-  Military  Training  Courses 

-  Promotion  Rate 

-  Leadership/Supervision  Rating  Composite 

-  Technical  Skill/Effort  Rating  Composite 

-  Overall  MOS  Rating  Composite 

-  Situational  Judgment  Test  Total  Score 

•  Personal  Discipline  (MPD) 

-  Discipl inary  Actions  (reversed) 

-  Personal  Discipline  Rating  Composite 

•  Physical  Fitness/Military  Bearing  (PFB) 

-  Physical  Readiness  Score 

-  Physical  Fitness/Bearing  Rating  Composite 

•  Training  and  Counseling  Subordinates  (TCS) 

-  Simulation  Exercise  -  Personal  Counseling  Content 

-  Simulation  Exercise  -  Personal  Counseling  Process 

-  Simulation  Exercise  -  Disciplinary  Content 

-  Simulation  Exercise  -  Disciplinary  Process 

-  Simulation  Exercise  -  Training 

•  Written  Methods  (WM) 

-  MOS-Specific  Knowledge 

-  Common  Soldiering  Knowledge 

-  Situational  Judgment  Test 

•  Ratings  Methods  (RM) 

-  Four  Army-Wide  Rating  Composites 

-  Overall  MOS  Rating  Composite 


Figure  1.9.  Relationship  of  Specific  Variables  to  Overall  Factors  in  the  CVII 
Performance  Model. 
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SUMMARY  OF  PROJECT  EFFORTS  FOR  YEAR  TWO 


As  described  in  the  annual  report  for  the  1991  fiscal  year  (Campbell  & 
Zook,  1994a),  year  two  was  a  period  of  score  development,  model  building,  and 
basic  validation  analyses  for  (a)  training  performance  (EOT),  (b)  first-tour 
performance  (LVI),  and  (c)  second-tour  performance  (CVII).  During  year  two, 
the  second-tour  longitudinal  data  collection  (LVII)  began  and  was  ongoing. 

Objectives 

The  specific  objectives  for  the  second-year  annual  report  were  as 
fol lows: 

(1)  Describe  the  development  of  alternative  scores  for  the  Assessment 
of  Background  and  Life  Experiences  (ABLE)  instrument. 

(2)  Describe  the  basic  validation  analyses  for  the  prediction  of 
performance  in  training. 

(3)  Describe  the  development  of  basic  scores  for  the  longitudinal 
sample  first-tour  performance  measures. 

(4)  Describe  the  replication/confirmation  of  the  first-tour  performance 
model  and  the  basic  Longitudinal  Validation  analyses  for  the 
Experimental  Predictor  Battery  against  first-tour  performance. 

(5)  Describe  the  basic  validation  analyses  for  the  prediction  of 
second-tour  performance,  using  the  CVII  sample. 

(6)  Report  the  results  of  a  preliminary  analysis  of  the  prediction  of 
second-tour  performance  from  first-tour  predictors  and  performance. 


Development  of  Alternative  ABLE  Factor  Composites 

As  part  of  Project  A,  and  based  on  the  results  of  an  extensive  review  of 
the  literature,  10  temperament  scales  had  been  developed  to  form  the  ABLE. 
These  constructs  were  selected  as  the  most  promising  for  predicting  perform¬ 
ance  in  Army  enlisted  occupational  specialties.  In  addition,  four  validity 
scales  were  included  to  detect  inaccuracies  in  self-reports  of  temperament  and 
a  self-report  measure  of  physical  condition  was  also  included  (see  Hough, 
Eaton,  Dunnette,  Kamp,  &  McCloy,  1990,  for  more  information  on  the  development 
of  ABLE).  To  develop  a  set  of  conceptually  meaningful  construct  (composite) 
scores,  Peterson  et  al.  U990)  carried  out  both  exploratory  and  confirmatory 
factor  analyses  on  the  correlation  among  the  content  scale  scores. 

The  resulting  seven  temperament  constructs  (composites)  and  associated 
ABLE  scales  are  shown  in  Table  1.1.  The  constructs  of  Dependability, 

Dominance  (Surgency),  Adjustment,  and  Cooperativeness  have  counterparts  in  the 
Big  Five  personality  dimensions  described  by  Norman  (1963)  and  Goldberg 
(1981).  Conversely,  Achievement  and  Internal  Control  are  not  in  the  Big  Five 
taxonomy,  but  were  among  the  strongest  predictors  of  job  performance  in  the 
Project  A  review  of  the  temperament  domain  (see  Hough,  1992,  for  more  details 
on  the  relationship  of  ABLE  constructs  to  the  Big  Five). 
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Table  1.1 


ABLE  Rational  Composites  and  Corresponding  Content  Scales 


_ Composite _ 

Achievement  Orientation 

Leadership  Potential 
Dependabi 1 ity 

Adjustment 
Cooperativeness 
Internal  Control 
Physical  Condition 


ABLE  Scale 

Self-Esteem 
Work  Orientation 
Energy  Level 

Dominance 

Traditional  Values 

Conscientiousness 

Nondelinquency 

Emotional  Stability 

Cooperativeness 

Internal  Control 

Physical  Condition 


As  noted  above,  a  rational/theoretical  approach  was  the  primary  method 
used  in  developing  ABLE.  An  alternative  empirical  procedure  emphasizes  the 
internal  covariance  structure  of  a  set  of  items  and  uses  factor  analytic 
methods.  Consequently,  during  year  two,  internal  scale  construction  methods 
were  used  to  increase,  through  homogeneous  keying,  the  internal  consistency  of 
ABLE  composites  and  to  decrease  their  intercorrelations. 

Results  from  factor  analyses  of  199  items  were  used  to  form  seven 
preliminary  composites.  These  composites  contained  99  items.  Next, 
correlations  between  the  remaining  content-type  items  (excluding  the  validity 
scale  items)  and  the  preliminary  factor  composites  were  examined  and  each 
remaining  item  was  assigned  to  the  composite  with  which  it  had  the  highest 
correlation.  The  seven  factor  composites  resulting  from  this  procedure  used 
168  items  and  are  called  the  ABLE-168  composites.  In  all,  125  items  were 
assigned  in  the  same  way  on  the  ABLE-168  composites  and  the  ABLE  rational 
composites. 

As  a  second  alternative,  an  item  was  retained  only  if  it  correlated  at 
least  .33  with  the  scale  for  which  it  was  assigned  and  had  a  higher 
correlation  with  its  own  composite  (by  .03)  than  any  other.  In  addition, 
several  items  that  added  only  minimally  to  internal  consistency  were  dropped. 
The  resulting  set  of  composites  had  a  total  of  114  items  and  is  called  the 
ABLE- 114  composites.  Eighty-nine  of  these  items  were  assigned  in  the  same  way 
on  ABLE-114  and  the  ABLE  rational  composites. 

The  three  scoring  methods  converged  to  yield  seven  similar  temperament 
constructs.  The  composites  measuring  the  same  constructs  were  very  highly 
correlated  (£  =  .88  to  1.0). 
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ABLE-114  composites  had  greater  discriminant  validity  than  either  the 
ABLE-168  factor  composites  or  the  ABLE  rational  composites.  The  average 
correlation  among  the  composites  (off-diagonal  elements)  was  .40  for  ABLE-114, 
and  .47  for  the  ABLE  rational  composites  and  ABLE-168. 

Table  1.2  shows  the  distribution  of  items  on  ABLE-168  and  ABLE-114  for 
each  of  the  ABLE  content  scales.  Items  outside  the  shaded  areas  were  assigned 
differently  on  the  rational  and  factor  composites.  There  is  much  overlap 
between  the  rational  and  factor  composites.  However,  approximately  25  percent 
of  item  assignments  for  the  factor  composites  were  different  from  those  used 
for  the  rational  composites.  Most  of  these  are  consistent  with  results  from 
previous  research  and/or  can  be  understood  on  the  basis  of  item  content. 

In  sum,  there  are  three  alternative  sets  of  ABLE  composites  measuring 
seven  temperament  constructs.  The  114-item  form  is  shorter  and  has  higher 
discriminant  validity  than  the  other  two  sets  of  composites,  with  little 
apparent  loss  of  reliability.  Subsequent  analyses  in  the  Career  Force  Project 
examine  the  criterion-related  validities  of  these  alternative  sets  of 
composites. 


Prediction  of  Performance  in  Training 

The  objectives  of  analyses  of  the  end-of-training  (EOT)  data  were  to: 


(1) 

Compute  the  validities  for  ASVAB  and  Experimental 
Battery  predictors  against  rating  measures  and  also 
paper-and-pencil  test  measures  of  training 
performance. 

(2) 

Compare  the  validities  of  four  alternative  sets  of 
ASVAB  scores. 

(3) 

Compare  the  validities  of  three  alternative  sets  of 
ABLE  scores. 

(4) 

Assess  the  incremental  validities  for  the  Experimental 
Battery  predictors  over  ASVAB. 

Procedure 

The  EOT  validation  analysis  consisted  of  the  following  steps: 

A)  Multiple  correlations  between  each  set  of  predictor  scores  and 

each  set  of  criterion  scores  were  computed  separately  by  MOS  and 
then  averaged  across  the  Batch  A  MOS  and  across  all  MOS. 
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Table  1.2 

Distribution  of  ABLE  Scale  Items  on  ABLE-168  and  ABLE-114  Factor  Composites 
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Noic.  ABLE-114  items  are  shown  in  parentheses.  Shaded  areas  indicate  convergence  between  the  rational  and  factor  composites. 


1)  The  ASVAB  predictor  set  was  represented  by: 

a)  The  nine  ASVAB  subtest  scores 

b)  The  four  ASVAB  factor  scores 

c)  The  Armed  Forces  Qualification  Test  (AFQT) 

d)  The  MOS-appropriate  Aptitude  Area  composite  score 

2)  The  ABLE  predictor  set  was  represented  by  three  sets  of 
scores: 

a)  The  seven  rational  scales 

b)  Seven  empirical  scales  that  retained  168  items 

c)  Seven  empirical  scales  that  retained  only  114  items 

3)  Each  of  the  other  predictor  sets  (i.e.,  spatial,  computer, 
AVOICE,  JOB)  was  represented  as  in  previous  analyses. 

All  results  were  adjusted  for  shrinkage  and  corrected  for 
multivariate  range  restriction. 

B)  Incremental  validity  was  computed  for  each  set  of  Experimental 
Battery  predictors  over  the  ASVAB. 

C)  Multiple  correlations  were  computed  between  each  set  of  predictor 
scores  and  a  "Peer  1"  rating,  a  "Peer  2"  rating,  a  supervisor 
rating,  and  various  combinations. 

Results 


To  summarize  the  principal  findings,  multiple  correlations  for  six 
predictor  sets  are  shown  in  Table  1.3;  the  incremental  validities  are 
summarized  in  Table  1.4.  In  general,  ASVAB  shows  high  validity  against  the 
school  knowledge  measures  and  the  relative  validities  for  the  four  ratings 
factors  are  as  would  be  expected  on  the  basis  of  the  factors.  The  ABLE  does 
not  predict  the  "will  do"  factors  quite  as  well  as  it  did  in  CVI  but  it 
predicts  the  "can  do"  factors  somewhat  better. 

These  results  indicate  that  the  level  of  validity  of  the  ASVAB  factors 
for  predicting  the  School  Knowledge  (SK)  test  scores  was  extremely  high, 
especially  for  the  Technical  (SK-Tech)  and  Total  (SK-Total)  scores.  Likewise, 
the  spatial  composite  and  the  computer  battery  produced  high  validities  for 
these  criteria. 

Results  from  other  analyses  indicate  that  peer  ratings  of  training 
performance  are  more  accurately  predicted  than  supervisor  ratings  of  training 
performance.  This  suggests  that  peer  ratings  may  be  more  valid  training 
measures  than  supervisor  ratings,  presumably  because,  in  training,  peers 
generally  have  greater  opportunity  to  observe  ratees  than  do  supervisors. 

This  comparison  is  confounded,  however,  by  the  greater  reliability  of  the  peer 
ratings  that  is,  at  least  in  part,  due  to  the  fact  that  they  are  based  on  more 
raters  per  ratee  than  are  the  supervisor  ratings.  Yet  analyses  at  the  1-rater 
level  corroborate  the  notion  that  the  peer  ratings  have  more  utility  than  the 
supervisor  ratings  for  assessing  training  performance. 


22 


F 


Table  1.3 


Mean  of  Multiple  Correlations  Computed  Within  Job  for  End-of-Training  Sample 
for  ASVAB  Factors,  Spatial,  Computer,  JOB,  ABLE  Rational  Composites,  and 
AVOICE  _ _ _ _ _ 


Criterion® 

MOS 

No.  of 
MOS^’ 

ASVAB 

Factors 

[4] 

Spatial 

[1] 

Computer 

[8] 

JOB 

[3] 

ABLE 

Comp. 

[7] 

AVOICE 

[8] 

Peer-ETS 

Batch  A 

11 

41  (07) 

35  (05) 

36  (05) 

24  (06) 

19  (09) 

22  (07) 

All  MOS 

22 

43  (13) 

37  (10) 

33  (14) 

23  (11) 

23  (12) 

23  (lO) 

Peer-MPD 

Batch  A 

11 

25  (04) 

22  (05) 

21  (05) 

09  (07) 

19  (05) 

11  (07) 

All  MOS 

22 

26  (11) 

22  (08) 

15  (10) 

12  (10) 

22  (10) 

09  (09) 

Peer-PFB 

Batch  A 

11 

14  (09) 

05  (06) 

11  (05) 

05  (05) 

29  (06) 

07  (07) 

All  MOS 

22 

19  (14) 

10  (11) 

12  (09) 

09  (12) 

26  (11) 

10  (lO) 

Peer-LEAD 

Batch  A 

11 

30  (10) 

24  (07) 

28  (07) 

18  (09) 

22  (09) 

17  (10) 

All  MOS 

22 

30  (16) 

26  (12) 

25  (16) 

20  (14) 

22  (12) 

16  (14) 

Supv-ETS 

Batch  A 

11 

21  (06) 

18  (05) 

17  (10) 

10  (08) 

09  (10) 

11  (10)  ■ 

All  MOS 

22 

27  (15) 

22  (11) 

18  (13) 

10  (10) 

11  (12) 

10  (lO) 

Supv-MPO 

Batch  A 

11 

13  (09) 

12  (07) 

11  (08) 

06  (06) 

05  (06) 

06  (06) 

All  MOS 

22 

16  (16) 

14  (11) 

10  (13) 

06  (08) 

05  (07) 

04  (06) 

Supv-PFB 

Batch  A 

11 

11  (07) 

09  (05) 

09  (08) 

06  (05) 

11  (09) 

07  (07) 

All  MOS 

22 

16  (15) 

13  (12) 

11  (15) 

05  (07) 

11  (11) 

05  (06) 

Supv-LEAD 

Batch  A 

11 

15  (10) 

14  (08) 

13  (10) 

08  (08) 

10  (11) 

08  (09) 

All  MOS 

22 

19  (17) 

17  (11) 

12  (12) 

11  (09) 

11  (12) 

07  (09) 

SK-Bas ic 

Batch  A 

9 

68  (06) 

57  (06) 

57  (06) 

38  (05) 

30  (07) 

37  (05) 

All  MOS 

20 

67  (08) 

58  (07) 

55  (14) 

36  (10) 

31  (14) 

37  (11) 

SK-Tech 

Batch  A 

11 

76  (05) 

63  (05) 

61  (05) 

41  (07) 

33  (05) 

44  (07) 

All  MOS 

22 

75  (06) 

62  (08) 

59  (06) 

38  (11) 

33  (13) 

40  (12) 

5K-Total 

Batch  A 

11 

78  (03) 

65  (04) 

64  (03) 

43  (07) 

34  (05) 

45  (06) 

All  MOS 

22  ■ 

77  (05) 

65  (07) 

62  (07) 

40  (11) 

35  (l4) 

42  (13) 

Note:  Corrected  for  range  restriction  and  adjusted  for  shrinkage  (Rozeboom,  1978,  formula  8).  Numbers  in 

brackets  are  the  numbers  of  predictor  scores  entering  prediction  equations.  Numbers  in  parentheses 
are  standard  deviations.  Decimals  omitted. 

^  ETS  =  Effort  and  Technical  Skill;  MPD  =  Maintaining  Personal  Discipline;  PFB  *  Physical  Fitness  and 
Military  Bearing;  LEAD  =  Leadership  Potential;  SK  =  School  Knowledge. 

Number  of  MOS  for  which  validities  were  corrputed. 
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Table  1.4 


Mean  of  Incremental  Correlations  Over  ASVAB  FactoY'S  Computed  Within  Job  for 
End-of-Training  Sample  for  Spatial,  Computer,  JOB,  ABLE  Rational  Composites, 

and  AVOICE  _ _ _ _ _ 


Criterion® 

MOS 

No.  of 
M0S‘’ 

A4 

ASVAB 

Factors 

[4] 

A4+ 

Spatial 

[5] 

A4+ 

Computer 

[12] 

A4+ 

JOB 

[7] 

A4+ 

ABLE 

Comp. 

[11] 

A4+ 

AVOICE 

[12] 

Batch  A 

11 

41  (07) 

42  (07) 

42  (06) 

41  (07) 

M  (06) 

41  (07) 

All  MOS 

22 

43  (13) 

42  (14) 

40  (16) 

42  (13) 

4b  (11) 

41  (14) 

pppp_Mpn 

Batch  A 

11 

25  (04) 

25  (05) 

24  (05) 

25  (05) 

34  (06) 

24  (07) 

All  MOS 

22 

26  (11) 

25  (11) 

22  (12) 

25  (12) 

M  (11) 

22  (11) 

pppr-PFR 

Batch  A 

11 

14  (09) 

13  (09) 

17  (07) 

15  (09) 

31  (09) 

15  (09) 

All  MOS 

22 

19  (14) 

18  (14) 

16  (12) 

19  (17) 

M  (14) 

18  (11) 

Pppr-[  FAD 

Batch  A 

11 

30  (10) 

30  (10) 

31  (08) 

30  (11) 

35  (09) 

29  (13) 

All  MOS 

22 

30  (16) 

30  (17) 

28  (18) 

n  (18) 

34  (15) 

28  (18) 

Supv-ETS 

Batch  A 

11 

21  (06) 

21  (07) 

19  (09) 

20  (06) 

19  (12) 

17  (12) 

All  MOS 

22 

(15) 

26  (15) 

24  (15) 

25  (15) 

25  (19) 

22  (16) 

Supv-MPD 

Batch  A 

11 

13  (09) 

12  (09) 

11  (09) 

11  (09) 

13  (11) 

11  (10) 

All  MOS 

22 

16  (16) 

16  (16) 

12  (17) 

14  (17) 

16  (16) 

11  (14) 

Supv~PFB 

Batch  A 

11 

11  (07) 

11  (07) 

10  (08) 

10  (07) 

16  (09) 

10  (09) 

All  MOS 

22 

16  (15) 

15  (14) 

12  (15) 

14  (13) 

18  (13) 

11  (13) 

Supv-LEAO 

Batch  A 

11 

15  (10) 

14  (10) 

14  (11) 

14  (10) 

16  (13) 

13  (12) 

All  MOS 

22 

19  (17) 

19  (17) 

15  (15) 

19  (16) 

20  (17) 

15  (15) 

SK-Bas i c 

Batch  A 

9 

68  (06) 

69  (06) 

68  (06) 

68  (06) 

68  (07) 

68  (06) 

All  MOS 

20 

67  (08) 

68  (08) 

65  (16) 

67  (09) 

66  (11) 

66  (10) 

SK-Tech 

Batch  A 

11 

76  (05) 

77  (05) 

77  (05) 

76  (05) 

76  (05) 

76  (05) 

All  MOS 

22 

75  (06) 

75  (06) 

75  (05) 

75  (06) 

75  (07) 

74  (07) 

SK-Tota 1 

Batch  A 

11 

78  (03) 

79  (03) 

79  (03) 

78  (03) 

79  (03) 

78  (04) 

All  MOS 

22 

77  (05) 

77  (05) 

77  (05) 

77  (05) 

77  (06) 

76  (06) 

Note;  Corrected  for  range  restriction  and  adjusted  for  shrinkage  (Rozeboom,  1978.  formula  8).  Numbers  in 
brackets  are  the  numbers  of  predictor  scores  entering  prediction  equations.  Numbers  in  parentheses 
are  standard  deviations.  Multiple  Rs  for  ASVAB  Factors  alone  are  in  italics.  Underlined  numbers 
denote  multiple  Rs  greater  than  for  ASVAB  Factors  alone.  Decimals  omitted. 

^  ETS  =  Effort  and  Technical  Skill;  MPO  =  Maintaining  Personal  Discipline;  PFB  =  Physical  Fitness  and 
Military  Bearing;  LEAD  =  Leadership  Potential;  SK  *  School  Knowledge. 

^  Number  of  MOS  for  which  validities  were  computed. 
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Further  analysis  showed  that  the  average  multiple  correlations  for  the 
four  different  sets  of  ASVAB  scores  differed  only  slightly  in  validity,  except 
that  the  peer  ratings  of  Physical  Fitness  (PFB)  were  better  predicted  by  the 
nine  subtests  and  the  four  factors.  However,  the  school  knowledge  test  scores 
were  predicted  somewhat  better  (about  three  to  five  points)  by  the  ASVAB 
subtests  and  factors  than  by  the  AFQT  or  Aptitude  Area  composites. 

Both  ABLE  and  AVOICE  predicted  the  knowledge-based  scores  quite  well. 

The  largest  incremental  validities  were  for  ABLE  over  ASVAB  when  predicting 
Personal  Discipline,  Fitness  and  Bearing,  and  Leadership. 

Finally,  there  were  virtually  no  differences  in  validities  for  the  three 
alternative  sets  of  ABLE  scores  although  the  ABLE-114  validities  were 
consistently  slightly  higher. 

Development  of  Basic  Scores  for  the  Longitudinal  Validation  (LVI) 

Performance  Measures 

In  1988  and  1989,  first-tour  criterion  measures  were  administered  to  the 
Longitudinal  Validation  sample  (LVI).  This  data  collection  was  conducted 
concurrently  with  the  administration  of  second-tour  criterion  measures  to  the 
Concurrent  Validation  sample  (CVII).  Before  the  LVI  performance  model 
development  and  subsequent  validation  analyses  could  begin,  it  was  necessary 
to  derive  basic  scores  for  each  of  the  individual  first-tour  job  performance 
measures.  Dealing  with  all  the  individual  scores  from  each  task  test,  each 
rating  scale,  and  each  administrative  index  was  simply  not  feasible  or 
desirable.  There  were  too  many,  and  the  reliabilities  of  the  individual  items 
or  scales  preserved  too  much  measurement  error  with  very  little  gain  in  total 
information.  Consequently,  the  full  array  of  scale  scores  was  aggregated  into 
a  smaller  set  of  basic  scores  for  each  measure. 

Table  1.5  lists  the  individual  measures  that  were  administered. 

Differences  Between  CVI  and  LVI  Performance  Measures 

The  3-year  time  period  between  CVI  and  LVI  raised  the  issue  that  for  the 
job  knowledge  and  hands-on  measures,  equipment  and/or  procedural  changes  would 
require  test  revisions,  and  changes  in  MOS  responsibilities  had  the  potential 
of  making  some  tasks  obsolete. 

Project  staff  identified  relevant  changes  so  that  the  appropriate 
revisions  could  be  made.  In  a  few  cases  where  an  entire  task  was  obsolete, 
the  task  was  dropped  without  replacement.  In  many  cases,  revisions  were 
simply  a  matter  of  replacing  outdated  terminology.  Updated  criterion  measures 
were  forwarded  to  the  MOS  proponents  for  a  currency  review  and  additional 
revisions  were  made  on  the  basis  of  this  review. 

While  there  was  considerable  interest  in  keeping  the  Combat  Performance 
Prediction  Scales,  project  staff  and  the  Scientific  Advisory  Group  agreed  that 
the  version  used  in  CVI  was  too  lengthy.  New  scales  were  field  tested  in 
conjunction  with  the  second-tour  criterion  measure  field  tests.  The  decision 
was  made  to  retain  the  original  summated  scale  format,  but  the  total  number  of 
items  was  reduced  from  40  to  19. 
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Table  1.5 

Measures  Administered  to  Soldiers  in  LVI  Sample 


MOS  in 

Batch  A:  Background  Information  Form 
Job  Knowledge  Tests 
Hands-On  Tests 
Army-Wide  Rating  Scales 
MOS-Specific  Rating  Scales 

Combat  Performance  Prediction  Scales  (males  only) 

Personnel  File  Form 

Army  Job  Satisfaction  Questionnaire 

Job  History  Questionnaire 

Physical  Requirements  Survey 

MOS  in 

Batch  Z:  Background  Information  Form 
School  Knowledge  Test 
Army-Wide  Rating  Scales 

Combat  Performance  Prediction  Scales  (males  only) 

Personnel  File  Form 

Army  Job  Satisfaction  Questionnaire 

Physical  Requirements  Survey 


Note.  Rating  scale  data  were  collected  from  both  supervisors  and  peers. 

The  Physical  Requirements  Survey  is  not  a  Career  Force  or  Project  A 
measure. 


The  self-report  form  for  gathering  information  on  administrative 
records  was  updated  by  reviewing  its  contents  with  officers  and  NCOs 
representing  the  Army  Personnel  Command  (PERSCOM).  The  form  was  altered  to 
allow  soldiers  to  report  an  M19  qualification  in  the  event  that  an  M16 
qualification  was  not  applicable.  Also,  three  awards  were  dropped  per 
guidance  from  PERSCOM. 

Task-level  ratings  were  deleted  from  the  array  of  Batch  A  first-tour 
criterion  measures  used  in  CVI.  The  Army-wide  and  MOS-specific  rating  scales 
were  retained  in  their  original  form. 

The  development  of  the  basic  scores  for  each  measure  was  based  on  the 
performance  data  collected  from  individuals  in  the  Batch  A  and  Batch  Z  MOS 
that  were  included  in  the  administration  of  first-tour  criterion  measures  in 
1988  and  1989.  The  Batch  A  MOS  were  the  same  as  those  studied  in  the 
Concurrent  Validation,  except  for  the  addition  of  19K  (Ml  Armor  Crewman). 

As  in  CVI,  the  Batch  A  MOS  differed  from  the  Batch  Z  MOS  in  the 
comprehensiveness  of  the  MOS-specific  criterion  measures  that  were  available 
for  administration.  MOS-specific  rating  scales,  hands-on  tests,  and  job 
knowledge  tests  were  administered  to  Batch  A  soldiers.  The  only  MOS-specific 
measure  available  for  administration  to  the  Batch  Z  soldiers  was  the  school 
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knowledge  test  that  had  been  developed  for  administration  at  the  end  of 
training.  The  school  knowledge  test  was  administered  to  the  Batch  Z  examinees 
as  a  surrogate  for  a  job  knowledge  test. 

Score  Development  for  Administrative  Indices 

Five  scores  were  computed  from  the  LVI  Personnel  File  Form:  (a) 
awards  and  memoranda/certificates  of  achievement,  (b)  Physical  Readiness  Test, 
(c)  M16  qualification,  (d)  Articles  15  and  flag  actions  (disciplinary 
actions),  and  (e)  promotion  rate. 

The  first  score  was  a  composite  of  "(a)  awards  and  decorations;  (b) 
memoranda  of  appreciation,  commendation,  or  achievement;  and  (c)  certificates 
of  appreciation,  commendation,  or  achievement.  The  last  score,  promotion 
rate,  was  derived  from  data  available  in  the  Army's  computerized  personnel 
records.  It  was  the  residual  of  pay  grade  regressed  on  time  in  service, 
adjusted  by  MOS. 

Basic  Score  for  the  Combat  Performance  Prediction  Ratings 

Principal  components  analyses  of  the  LVI/CVII  Combat  Scale  data 
indicated  the  presence  of  two  factors.  The  second  factor,  however,  was 
defined  by  the  three  negatively  worded  items.  Given  that  the  second  factor 
was  probably  not  substantively  distinct  from  the  first,  a  single  total  score 
(with  the  negatively  worded  items  reverse-scored)  was  calculated  for  the 
Combat  Scale  ratings.  (The  two  factors  were  essentially  the  same  as  those 
found  in  CVI,  where  two  Combat  Scale  scores  were  derived.) 

Development  of  Basic  Scores  for  the  First-Tour  Performance  Rating  Scales 

The  Army-wide  rating  scales  include  12  dimensions  of  soldier 
effectiveness  that  are  important  regardless  of  soldiers'  MOS.  MOS-specific 
rating  scales  were  developed  for  each  of  the  nine  Batch  A  MOS,  and  these 
rating  scales  include  between  7  and  13  dimensions  of  MOS-specific  performance. 

Principal  factor  analyses  with  varimax  rotation  were  conducted  on  the 
Army-wide  ratings  (across  all  MOS),  for  supervisor  and  peer  ratings  separately 
and  pooled  together.  The  pooled  ratings  were  computed  by  averaging  the  mean 
peer  rating  and  one  supervisor  rating  for  those  soldiers  who  had  at  least  one 
peer  rating  and  one  supervisor  rating.  Because  previous  analyses  (using  the 
CVI  sample)  showed  that  a  single  factor  was  sufficient  to  account  for  the 
majority  of  the  variance  in  the  MOS-specific  ratings,  factor  analyses  were  not 
conducted  for  the  MOS-specific  rating  data. 

Table  1.6  shows  the  three-factor,  rotated  solutions  for  the  pooled 
peer/supervisor  ratings.  These  data  demonstrate  the  remarkable  similarity  of 
the  rotated  factor  structures  for  the  CVI  and  LVI  samples.  It  is  worth  noting 
that  these  same  three  factors  were  also  obtained  in  factor  analyses  of 
performance  rating  data  for  a  sample  of  950  second-tour  soldiers,  which  was 
collected  using  a  set  of  rating  scales  very  similar  to  those  used  to  collect 
the  present  data  (Campbell  &  Zook,  1990). 
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Table  1.6 


Comparison  of  LVI  and  CVI 
Peer/Supervisor  Ratings 

Army-Wide  Factor  Analysis^  Results:  Pooled 

Factor  Loadinqs 

ILVI/CVI) 

Dimension 

1 

2 

3 

Technical  Knowledge/Skill 

.67/. 71 

.30/. 28 

.38/. 30 

Leadership 

.65/. 69 

.34/. 30 

.447.37 

Effort 

.66/. 69 

.477.43 

.327.26 

Self-Development 

.52/. 57 

.427.38 

.467.38 

Maintaining  Equipment 

.507.54 

.41/. 34 

.41/. 35 

Following  Regulations 

.39/. 41 

.737.69 

.31/. 30 

Self-Control 

.19/. 22 

.657.63 

.20/. 20 

Integrity 

.44/. 50 

.667.59 

.30/. 28 

Mi  1 itary  Bearing 

.31/. 32 

.357.32 

.577.57 

Physical  Fitness 

.24/. 21 

.16/. 15 

.497.49 

Percent  Common  Variance 

37.7/44.9 

36.6/32.7 

25.6/22.4 

Note.  Sample  size  is  7,919  for  LVI  and 

8,642  for  CVI. 

^  Principal  factor  analysis,  varimax  rotation. 

Computed  by  averaging  the  mean  peer  rating  and  the  mean  supervisor  rating. 


For  both  the  Ar*liiy-wide  and  MOS-specific  rating  scales,  the  mean, 
variability,  and  reliability  of  the  peer,  supervisor,  and  pooled  peer/ 
supervisor  ratings  appear  quite  acceptable  and  are  comparable  to  what  was 
found  in  the  CVI  research.  Factor  analyses  of  the  Army-wide  ratings  showed 
that  the  three-factor  CVI  solution  was  replicated  in  the  present  data. 
Accordingly,  the  three  composites  shown  in  Teble  1.7,  elong  with  the  overall 
effectiveness  rating,  were  used  as  the  basic  scores  for  the  Army-wide  rating 
data. 
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Table  1.7 


Composition  and  Definition  of  LVI  Army-Wide  Rating  Composites _ 

Percent  Common 
Variance  Accounted 
For  by  Relevant 

Factor  Name  and  Definition  Factor^  (LVI/CVI)  Dimensions  Included 


1.  Technical  Skills  and  Job  37.8/44.9 

Effort: 

Exerting  effort  over  the  full 
range  of  job  tasks;  engaging  in 
training  or  other  development 
activities  to  increase 
proficiency;  persevering  under 
dangerous  or  adverse  conditions; 
and  demonstrating  leadership  and 
support  toward  peers. 

2.  Personal  Discipline:  36.6/32.7 

Adhering  to  Army  rules  and 
regulations;  exercising  self- 
control;  demonstrating  integrity 
in  day-to-day  behavior;  and  not 
causing  disciplinary  problems. 

3.  Physical  Fitness/Military  25.6/22.4 

Bearing: 

Maintaining  an  appropriate 
military  appearance  and  bearing 
and  staying  in  good  physical 
condition. 


Technical  Knowledge/ 
Skill  Leadership 
Effort 

Self-Development 
Maintaining  Equipment 


Following  Regulations 

Self-Control 

Integrity 


Military  Bearing 
Physical  Fitness 


®  Factor  analysis  of  pooled  peer/supervisor  ratings. 

Development  of  Basic  Scores  for  Hands-On  Performance  and  Job  Knowledge 
Measures 

As  the  first  step  in  replicating  the  CVI  procedures  for  constructing  the 
basic  scores,  tasks  were  clustered  into  Functional  Categories  as  described  in 
the  Project  A  annual  report  for  1986  (Campbell,  1987). 

Following  the  procedures  developed  with  the  CVI  data,  tasks  were  also 
sorted  into  six  higher  level  groups  referred  to  as  Task  Factors  (Communica¬ 
tion,  Vehicles,  Basic  Techniques,  Identify  Targets,  Technical,  and  Safety/ 
Survival)  and  known  as  CVBITS.  Tasks  were  also  combined  into  just  two  groups: 
General  (i.e..  Army-wide)  and  MOS-specif ic. 
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In  general,  the  grouping  schemes  are  hierarchical:  Tasks  (the  lowest 
level)  are  placed  in  Functional  Categories,  the  Functional  Categories  (level 
two)  are  aggregated  to  form  the  six  Task  Factors  (level  three),  and  Task 

Factors  are  then  aggregated  to  form  the  two  Task  Constructs  (level  four),  as 

diagrammed  in  Figure  1.10. 

For  the  LVI  data,  confirmatory  factor  analyses  were  conducted  to  assess 
the  fit  of  alternative  levels  of  score  aggregation.  These  analyses  served  two 
purposes:  They  were  used  to  assess  the  relative  merits  of  each  model  and  to 

corroborate  the  CVI  decision  to  use  the  six  task  factor  scores  (CVBITS).  The 

analysis  required  the  computation  of  separate  tests  of  goodness  of  fit  for 
hands-on  and  job  knowledge  test  data,  for  each  of  the  10  MOS,  on  each  of  three 
competing  models.  The  three  models  tested  were:  a  one-factor  model, 
postulating  the  existence  of  a  single  factor  in  the  data;  a  two-factor  model, 
proposing  the  Basic  and  the  Technical  Task  Constructs;  and  a  three-to-six- 
factor  model  (the  number  of  factors  varying  among  MOS  and  test  method),  using 
the  Task  Factors.  Examination  of  the  results  from  LVI  argues  for  the 
retention  of  the  six  Task  Factor  scores  for  both  the  Hands-On  and  Job 
Knowledge  measures. 

Final  Array  of  LVI  Basic  Performance  Scores 

A  summary  list  of  the  basic  performance  scores  produced  by  the  analyses 
summarized  above  is  given  in  Figure  1.11.  These  are  the  scores  that  were  put 
through  the  final  editing  and  score  imputation  procedures  for  the  LVI  data 
file.  The  scores  that  formed  the  basis  for  the  confirmatory  tests  of  the  LVI 
model  of  first-tour  job  performance  were  also  drawn  from  this  array. 

The  LVI  Data  File:  Final  Data  Editing  and  Score  Imputation 

The  Longitudinal  Validation  First-Tour  (LVI)  data  were  collected  from 
11,266  soldiers  in  21  MOS  --  6,815  from  Batch  A  MOS  and  4,451  from  Batch  Z 
MOS.  Extensive  efforts  were  made  to  collect  complete  information  from  each 
examinee  for  all  instruments.  However,  as  with  all  data  collection  exercises, 
circumstances  precluded  complete  success.  The  final  counts  of  soldiers  for 
whom  data  were  analyzed  for  each  instrument  are  given  in  Tables  1.8  and  1.9 
for  Batch  A  and  Batch  Z  MOS,  respectively. 

Data  for  each  performance  measure  were  processed  individually.  After 
processing  was  completed  for  these  individual  measures,  they  were  combined  so 
that  all  LVI  data  for  each  examinee  were  included  in  a  single  file.  The  data 
were  combined  separately  by  MOS.  When  the  data  were  combined,  basic  scores 
were  calculated  for  the  individual  performance  measures.  Table  1.10  shows  the 
amount  of  missing  data  for  the  final  set  of  basic  criterion  scores. 

In  addition  to  the  performance  data,  missing  Longitudinal  Validation 
predictor  data  were  also  imputed.  For  a  complete  description  of  the  editing 
process  used  on  the  predictor  data,  see  the  1990  annual  report.  The  bulk  of 
the  editing  process  was  accomplished  during  FY90,  but  additional  work  was  done 
during  FY91.  The  amounts  of  missing  data  for  each  score  on  each  paper-and- 
pencil  and  each  computerized  measure  are  shown  in  Tables  1.11  and  1.12. 
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Task 

Examples 


Functional 

Categories 


Task 

Factors 


Task 

Constructs 


Safety/Survival 


Basic  Soldiering 


Communications - 


Identify 


Vehicles 


Technical 


General 


MOS- 

Specific 


Note.  The  Task  Factors  correspond  to  the  six  task  groups  known  as  CVBITS.  The  Task  Constructs  termed  General  and  MOS-Specific 
refer  to  the  same  constructs  that  have  previously  been  called  Basic  and  Technical,  or  Common  and  Technical. 


Figure  1.10.  Hierarchical  Relationships  Among  Functional  Categories, 
Task  Factors,  and  Task  Constructs 
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Hands-On  Performance  Test 

1.  Safety/survival  performance  score 

2.  General  (common)  task  performance  score 

3.  Communication  performance  score 

4.  Vehicles  performance  score 

5.  MOS-specific  task  performance  score 

Job  Knowledge  Test 

6.  Safety/survival  knowledge  score 

7.  General  (common)  task  knowledge  score 

8.  Communication  knowledge  score 

9.  Identify  targets  knowledge  score 

10.  Vehicles  knowledge  score 

11.  MOS-specific  task  knowledge  score 

Army-Wide  Rating  Scales 

12.  Overall  effectiveness  rating 

13.  Technical  skill  and  effort  composite 

14.  Personal  discipline  composite 

15.  Physical  fitness/military  bearing  composite 

MQS-Specific  Rating  Scales 

16.  Overall  MOS  composite 

Combat  Performance  Prediction  Scales 

17.  Overall  Combat  Prediction  scale  composite  (available  for  males  only) 
Personnel  File  Form 

18.  Awards  and  Certificates 

19.  Disciplinary  Actions  (Articles  15  and  Flag  Actions) 

20.  Physical  Readiness 

21.  M16  Qualification 

22.  Promotion  Rate 


Figure  1.11.  Summary  List  of  LVI  Basic  Criterion  Scores 
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Table  1.8 


LVI  Sample  Sizes  for  Performance  Measures  for  Batch  A  MOS 


Job  Army- 


MOS 

N 

Hands- 

On 

Knowl¬ 

edge 

Wide 

Ratings 

MOS 

Ratings 

Combat 

Ratings 

Personnel 

File 

Combined 

Criteria^ 

IIB 

Infantryman 

909 

890 

895 

899 

899 

898 

906 

907 

13B 

Cannon  Crewmember 

916 

773 

810 

897 

897 

897 

916 

916 

19E 

M60  Armor  Crewman 

249 

243 

248 

241 

241 

241 

249 

249 

19K 

Ml  Armor  Crewman 

824 

749 

812 

782 

778 

782 

819 

825 

31C 

Single  Channel  Radio  Operator 

529 

446 

504 

497 

481 

442 

527 

529 

63B 

Light- Wheel  Vehicle  Mechanic 

752 

624 

723 

728 

719 

666 

750 

752 

71L 

Administrative  Specialist 

678 

641 

664 

634 

626 

199 

675 

678 

88M 

Motor  Transport  Operator 

682 

588 

674 

666 

663 

479 

680 

682 

91A 

Medical  Specialist 

824 

794 

798 

807 

797 

670 

818 

824 

95B 

Military  Police 

452 

444 

446 

451 

450 

366 

452 

452 

Total 

6,815 

6,192 

6,574 

6,602 

6,547 

5,640 

6,792 

6,814 

®  Combined  Criteria  include  Hands-On,  Job  Knowledge,  Army-Wide  Ratings,  MOS  Ratings,  and  Personnel 
File  Form. 


Table  1.9 


LVI 

Sample  Sizes  for  Performance 

Measures 

for  Batch 

Z  MOS 

MOS 

N 

Job 

Knowledge 

Army- 

Wide 

Ratings 

Combat 

Ratings 

Personnel 

File 

12B 

Combat  Engineer 

841 

840 

827 

827 

838 

16S 

MANPADS  Crewman 

472 

471 

468 

468 

472 

27E 

Tow/Dragon  Repairer 

90 

90 

89 

84 

90 

29E 

Comm. -Electronics  Radio  Repa 

irer  112 

111 

106 

101 

111 

51B 

Carpentry/Masonry  Specialist 

213 

212 

193 

190 

212 

54B 

NBC  Specialist 

499 

498 

492 

462 

498 

55B 

Ammunition  Specialist 

279 

279 

269 

243 

279 

67N 

Utility  Helicopter  Repairer 

197 

194 

193 

192 

197 

76Y 

Unit  Supply  Specialist 

788 

788 

734 

616 

787 

94B 

Food  Service  Specialist 

832 

932 

818 

717 

931 

96B 

Intelligence  Analyst 

128 

128 

122 

103 

128 

Total 

4,451 

4,443 

4,311 

4,003 

4,443 
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LVI  Combined  Criteria  Data:  Percentage  of  Missing  Data  for  Basic  Scores  by  MOS _ 

Criteria  HB  13B  19E  19K  31C  63B  71L  88M  91A  95B 
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Note.  -  indicates  that  the  particular  score  was  not  calculated  for  that  MOS. 


Table  1.11 


LVI  Predictor  Data:  Amount  of  Missing  Data  for  Paper-and-Penci 1  Scale  Scores 


LVI  rreaiCLOr  udtd:  MINUUHL  uj  IJUI-U 

Score 

Not  Missing 

Missing 

Assembling  Objects  -  Number  Correct 

49,042 

366 

Map  -  Number  Correct 

49,047 

361 

Maze  -  Number  Correct 

49,052 

356 

Object  Rotation  -  Number  Correct 

49,103 

305 

Orientation  -  Number  Correct 

49,072 

336 

Reasoning  -  Number  Correct 

49,103 

305 

JOB  Scale  1  -  Pride 

46,525 

2,883 

JOB  Scale  2  -  Job  Security/Comfort 

46,634 

2,774 

JOB  Scale  3  -  Serving  Others 

46,295 

3,113 

JOB  Scale  4  -  Job  Autonomy 

46,037 

3,371 

JOB  Scale  5  -  Routine 

45,975 

3,433 

JOB  Scale  6  -  Ambition 

46,058 

3,350 

ABLE  Scale  1  -  Emotional  Stability 

44,264 

5,144 

ABLE  Scale  2  -  Self-Esteem 

44,247 

5 ,  Ibl 

ABLE  Scale  3  -  Cooperativeness 

44,258 

5,150 

ABLE  Scale  4  -  Conscientiousness 

44,199 

5,209 

ABLE  Scale  5  -  Nondelinquency 

44,228 

5,180 

ABLE  Scale  6  -  Traditional  Values 

44,190 

5,218 

ABLE  Scale  7  -  Work  Orientation 

44,260 

5,148 

ABLE  Scale  8  -  Internal  Control 

44,254 

5,154 

ABLE  Scale  9  -  Energy  Level 

44,217 

5,191 

ABLE  Scale  10  -  Dominance 

44,246 

5,162 

ABLE  Scale  11  -  Physical  Condition 

44,264 

5,144 

AVOICE  Scale  1  -  Clerical/Administrative 

45,477 

3,931 

AVOICE  Scale  2  -  Mechanics 

45,941 

3,467 

AVOICE  Scale  3  -  Heavy  Construction 

45,851 

3 , 557 

AVOICE  Scale  4  -  Electronics 

45,922 

3,486 

AVOICE  Scale  5  -  Combat 

45,939 

3,469 

AVOICE  Scale  6  -  Medical  Services 

45,545 

3 ,863 

AVOICE  Scale  7  -  Rugged  Individualism 

45,944 

3,464 

AVOICE  Scale  8  -  Leadership/Guidance 

45,508 

3,900 

AVOICE  Scale  9  -  Law  Enforcement 

45,958 

3,450 

AVOICE  Scale  10  -  Food  Service  Professional 

45,916 

3,492 

AVOICE  Scale  11-  Firearms  Enthusiast 

45,942 

3,466 

AVOICE  Scale  12  -  Science/Chemical 

45,970 

3,438 

AVOICE  Scale  13  -  Drafting 

45,976 

3,432 

AVOICE  Scale  14  -  Audiographics 

45,452 

3,956 

AVOICE  Scale  15  -  Aesthetics 

45,279 

4,129 

AVOICE  Scale  16  -  Computers 

45,554 

3,854 

AVOICE  Scale  17  -  Food  Service  Employee 

45,965 

3,443 

AVOICE  Scale  18  -  Mathematics 

45,691 

3,717 

AVOICE  Scale  19  -  Electronic  Communications 

45,602 

3,806 

AVOICE  Scale  20  -  Warehousing/Shipping 

45,963 

3,445 

AVOICE  Scale  21  -  Fire  Protection 

45,972 

3,436 

AVOICE  Scale  22  -  Vehicle  Operator 

45,971 

3,437 

35 


Table  1.12 


LVl  Predictor  Data^  Amount  of  Missing  Data  for  Computer-Administered  Scale 
Scores  _ ^ _ _ _ _ _ 


Score 


Not 

Missing  Missing 


Target  Identification  -  Mean  of  Clipped  Decision  Time 
Target  Identification  -  Proportion  Correct 

Number  Memory  -  Mean  of  Clipped  Operation  Means 
Number  Memory  -  Proportion  Correct 

Target  Track  1  -  Mean  Log  (Distance+1) 

Target  Track  2  -  Mean  Log  (Distance+1) 

Cannon  Shoot  -  Mean  Absolute  Time  Discrepancy 

Target  Shoot  -  Mean  Log  (Distance+1) 

Mean  of  Median  Movement  Times  across  5  tests 

Simple  Reaction  Time  -  Median  Decision  Time 
Simple  Reaction  Time  -  Proportion  Correct 

Choice  Reaction  Time  -  Median  Decision  Time 
Choice  Reaction  Time  -  Proportion  Correct 

Perceptual  Speed/Accuracy  -  Mean  of  Clipped  Decision  Time 
Perceptual  Speed/Accuracy  -  Proportion  Correct 

Short-Term  Memory  -  Mean  of  Clipped  Decision  Time 
Short-Term  Memory  -  Proportion  Correct 


38,401 

513 

38,404 

510 

38,324 

590 

38,353 

561 

38,825 

89 

38,793 

121 

38,603 

311 

37,477 

1,437 

37,863 

1,051 

38,747 

167 

38,747 

167 

38,856 

58 

38,856 

58 

38,703 

211 

38,734 

180 

38,483 

431 

38,490 

424 

An  imputation  procedure  known  as  PROC  IMPUTE  (Wise  &  McLaughlin,  1980) 
was  developed  that  used  existing  data  to  estimate  values  for  missing  data. 

This  procedure  was  also  used  in  the  CVI  analyses  (Wise,  McHenry,  &  Young, 
1986).  The  decision  rules  used  in  the  CVI  analyses  were  replicated  in  the  LVI 
analyses  as  closely  as  possible. 

PROC  IMPUTE  uses  regression  estimates  to  predict  missing  values.  Each 
missing  value  is  predicted  from  other  values  for  the  subject  in  question  so 
that  individual  differences  are  retained.  The  regression  coefficient  and 
intercept  vary  from  item  to  item  so  that  differences  in  item  difficulty  are 
also  reflected  in  the  predicted  values.  PROC  IMPUTE  also  adds  a  random 
variable  with  variance  equal  to  the  error  of  estimate  for  predicting  the 
missing  value. 

The  results  of  the  imputation  were  examined  at  two  levels.  First,  after 
each  PROC  IMPUTE  run,  the  program  output  was  inspected.  Second,  the  pre¬ 
imputed  and  the  post-imputed  data  sets  were  compared  for  each  MOS  (a)  after 
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the  hends-on  score  level  imputation,  and  (b)  after  the  criterion  construct 
level  imputation. 

The  means  and  variances  of  the  pre-  and  post- imputation  results  for  the 
hands-on  data  for  each  MOS  were  found  to  be  virtually  identical.  Imputation 
also  made  virtually  no  difference  in  the  magnitude  of  the  intercorrelations 
among  the  criterion  scores  that  were  used  to  create  the  performance  factor 
scores  in  the  validation  analyses.  These  results  are  similar  to  those 
obtained  earlier  from  the  CVI  imputation  (Wise  et  al.,  1986). 

Development  of  the  LVI  First-Tour  Performance  Model 

A  latent  factor  model  of  first-tour  performance,  developed  using  data 
from  the  Project  A  Concurrent  Validation  (CVI)  sample,  has  been  described  by 
Campbell,  McHenry,  and  Wise  (1990).  This  model  included  the  now  familiar  five 
performance  factors--Core  Technical  Proficiency  (CTP),  General  Soldiering 
Proficiency  (GSP),  Effort  and  Leadership  (ELS),  Maintaining  Personal 
Discipline  (MPD),  and  Physical  Fitness  and  Military  Bearing  (PFB)--  and  two 
measurement  method  factors,  a  Ratings  method  factor  and  a  Paper-and-Penci 1 
Test  method  factor.  During  year  two,  the  CVI  model  was  subjected  to  a 
confirmatory  analysis,  using  first-tour  performance  data  collected  from  the 
Longitudinal  Validation  (LVI)  sample.  Additionally,  comparative  analyses 
aimed  at  evaluating  more  parsimonious  models  of  first-tour  performance  were 
carried  out. 

An  earlier  section  summarized  how  each  of  the  major  sets  of  performance 
measures  was  reduced  from  a  large  number  of  item,  task,  or  individual  scale 
scores  to  a  smaller  set  of  factor  or  category  scores.  The  results  of  this 
first  level  of  aggregation  have  been  referred  to  as  the  "basic"  array  of 
criterion  scores,  summarized  in  Figure  1.11.  These  included  the  scores  that 
were  used  in  the  modeling  analyses  described  below. 

Altogether,  the  LVI  first-tour  performance  measures  were  reduced  to  20 
basic  scores.  However,  because  MOS  differ  in  their  task  content,  not  all  20 
variables  were  scored  in  each  MOS,,  and  there  was  some  slight  variation  in  the 
number  of  variables  used  in  the  subsequent  analyses. 

To  test  the  fit  of  the  different  models  to  the  LVI  data,  confirmatory 
factor-analytic  techniques  were  applied  to  each  MOS  individually,  using 
LISREL  7  (Joreskog  &  Sdrbom,  1989).  The  first  alternative  five-factor  model 
was  developed  using  CVI  data.  After  the  fit  of  the  five-factor  model  was 
assessed  in  each  MOS,. four  reduced  models  (all  nested  within  the  five-factor 
model)  were  examined.  Finally,  as  had  been  done  in  the  original  CVI  analyses, 
the  five-factor  model  was  applied  to  the  Batch  A  MOS  simultaneously  (using 
LISREL 's  multigroups  option).  The  fit  statistics  (e.g.,  root  mean-square 
residuals  [RMSRs])  of  the  five-factor  model  for  each  MOS  in  the  LVI  and  CVI 
samples  were  very  similar.  In  fact,  for  three  of  the  MOS  (IIB,  13B,  and  71L), 
the  RMSRs  for  the  LVI  data  were  smaller  than  those  for  the  CVI  data.  These 
results  indicate  that  the  model  developed  using  the  CVI  data  does  fit  the  LVI 
data  quite  wel 1 . 

Four  reduced  models  were  also  examined  using  the  LVI  data.  For  the 
four-factor  model .  the  Core  Technical  Proficiency  and  General  Soldiering 
Proficiency  performance  factors  were  collapsed  into  a  single  "can  do" 
performance  factor.  The  three-factor  model  retained  the  "can  do"  performance 
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factor  of  the  four-factor  model,  but  also  collapsed  the  Effort  and  Leadership 
and  Maintaining  Personal  Discipline  performance  factors  into  a  "will  do" 
performance  factor.  For  the  two-factor  model ,  the  can  do  performance  factor 
was  retained;  however,  the  Physical  Fitness  and  Military  Bearing  performance 
factor  became  part  of  the  "will  do"  performance  factor.  Finally,  for  the  one- 
factor  model,  the  "can  do"  and  "will  do"  performance  factors,  or  equivalently, 
the  five  original  performance  factors,  were  collapsed  into  a  single 
performance  factor. 

The  chi-square  statistics  and  RMSRs,  respectively,  for  the  four  reduced 
models,  as  well  as  for  the  five-factor  model,  indicate  that  the  four-  and 
five-factor  models  fit  the  LVI  data  well,  while  the  one-,  two-,  and  three- 
factor  models  fit  less  well.  The  results  also  indicated  that  the  parameter 
estimates  for  the  five-factor  model  were  generally  similar  across  the  10  MOS. 
The  final  step  was  to  determine  whether  the  variation  in  some  of  these 
parameters  could  be  attributed  to  sampling  variation.  To  do  this  (as 
described  earlier),  the  following  were  specified  to  be  invariant  across  jobs; 
(a)  the  correlations  among  performance  factors,  (b)  the  loadings  of  all  the 
Army-wide  measures  on  the  performance  factors  and  on  the  rating  method  factor, 

(c)  the  loadings  of  the  MOS-specific  score  on  the  rating  method  factor,  and 

(d)  the  uniqueness  coefficients  for  the  Army-wide  measures. 

The  results  indicated  that  the  fit  of  the  five-factor  model  is  not  as 
good  when  the  parameters  listed  above  are  constrained  to  be  equal  across  the 
10  jobs.  Still,  the  root  mean-square  residuals  associated  with  the  across-MOS 
model  are  not  substantially  greater  than  those  for  the  within- job  analyses. 
(The  average  RMSR  for  the  across-MOS  model  is  .0676;  the  average  for  the 
within-MOS  models  is  .0585.) 

To  create  criterion  construct  scores  for  use  in  validation  analyses,  the 
scoring  procedures  were  based  on  the  five-factor  model.  Although  the  four- 
factor  model  has  the  advantage  of  greater  parsimony,  the  five-factor  model 
offered  the  advantage  of  corresponding  to  the  criterion  constructs  generated 
in  the  CVI  validation  analyses.  Table  1.13  shows  the  mapping  of  the  basic 
scores  on  the  five  performance  factors.  As  with  the  CVI  data,  five  residual 
scores,  corresponding  to  the  five  criterion  constructs,  were  also  created. 

The  five  "raw"  criterion  construct  scores,  the  five  residual  criterion 
construct  scores,  the  total  rating  and  job  knowledge  scores,  and  the  total 
score  derived  from  the  hands-on  test  were  used  to  generate  a  13  x,13  matrix  of 
criterion  intercorrelations  for  each  MOS  in  Batch  A.  The  averages  of  these 
correlations  are  reported  in  Table  1.14.  These  results  are  very  similar  to 
the  correlations  that  were  reported  by  Campbell  et  al.  (1990)  for  the  CVI 
sample. 

Basic  Validation  Results  for  the  LVI  Sample 

The  LVI  validation  results  were  based  on  two  different  sample  editing 
strategies.  The  first  required  complete  data  for  all  predictor  composites,  as 
well  as  for  the  ASVAB,  and  for  each  performance  factor;  this  sample  is 
referred  to  as  the  "listwise  deletion"  sample.  In  the  alternative  strategy, 
called  setwise  deletion,  a  separate  validation  sample  was  identified  for  each 
set  of  predictors  in  the  Experimental  Battery. 
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Table  1.13 

Mapp i ng  of  LVII  Performance  Measures  Onto  Latent  Performance  Factors 


on 

on 

on 

c 

CD 

L- 

•r- 

1— ' 

O 

4-J 

03 

+-> 

03 

O 

u 

on 

03 

LL 

“O 

o 

JC 

+J 

c: 

CD 

1 

CD 

on 

CD 

-M 

on  -M 

O  T3 

on 

•1 — 

C 

CD 

CD 

X 

' — 

1— 

<U 

CJ 

c: 

rO 

e 

u 

o 

M- 

OJ 

CL 


cn 

c 


r— 

03  cr»  L- 
cj  on  03  . 
•r-  O)  c 


cn 

c 


03 


OJ 

- —  C 
03  -r- 
C  ' — 
O  Ol 

(/)  -r- 

S-.  O 
O)  on 
CL  •«- 
Q 


-O  CL 

c: 

03  x: 
oo 

-M  S- 
L-  CD 
O  “O 
03 

M-  (D 
LU  _J 


cn 

c 


u 


03  ,  CD 
S-  ^ 


O 

on 


S- 

CL 


• —  u 

03  C 
CD 

o|.^ 

^  U'+- 

OJ  o 

h—  L- 
Q- 


X  X  X  X 


X  X  X  X  X  X 


X 


X 


X  X 


X 


X  X  X  X 


X  X  X  X  X 


CD 

+-» 

*r-  <D 
LO  +-> 

o 

CD.  on  (D  CD 


(/) 

CD  on 
4->  t/j  on 


E 

O 

4-> 

CD 

03  CD  cim 

03 

O 

CL'-- 

*»— 

-L> 

CJ  C  03 

QC 

S-  r— 

s- 

CD 

f— 

c^ 

E  on 

on 

•1— 

•  1 —  *1—  1 — 

CD 

(D  03 

CD 

on 

03 

o  o 

O 

on 

“O  Li_ 

-4-> 

CD 

C  > 

c 

i- 

> 

-M 

C_3  CL 

CIL 

O 

•I-  03 

03 

T3 

<0 

CD 

o  “O  •»“ 

o 

-o 

03 

•  f— 

S- 

E 

E 

CL 

-I-)  CD  lD 

CL 

•t— 

h— 

> 

o 

CD  O 

O 

E 

u  t— 1 

o 

-♦->  O  L, 

+-> 

O 

s- 

M- 

C  C.J 

CJ 

o 

CD 

C 

i 

fi 

■ —  03  C>0  Z3 

, — 

03 

LO 

-M 

13 

M- 

•1— 

CJ 

c_)  < —  on 

o 

X 

c/n 

03  CJ  on  c>o 

03 

o 

on 

03 

c>o 

LlJ 

1 —  on 

• — 

03  CD 

E 

CJ  •»—  CD  ■ — 

u 

•f— 

CD 

1 — 

CD 

O-  t/n 

* — 

cn 

on  CJ  < — 

4J 

s- 

c 

•r-  C:  < —  03  >% 

*1 — 

c 

03 

S- 

f— 

•r~  CD 

03 

c: 

X3  *«"  CJ 

O 

<c 

o 

C  O  CJ  S-  4-> 

C 

o 

CJ 

s- 

.o 

-M 

1 — 

CJ  c 

L- 

U  on  ‘r- 

E 

^  E  *r-  CD  CD 

E 

-1-’ 

CD 

1— 

CD 

*»— 

on  4-> 

CD 

-L> 

ro  >> 

O 

II 

c 

CJE-CCM- 

CJ 

E 

c 

M- 

X 

•f“  -f— 

> 

03 

^  b 

c. 

CD 

CD  O  CD  CD  03 

CD 

o 

CD 

CD 

Q 

03 

c/n 

Q  Ll- 

O 

QC 

«cC  CL  <C 

CL 

CQ 

-4-J 

1 —  c«)  >  ci3  c/n 

h— 

c_) 

> 

CD 

►-H 

c/n 

12 

cn 

cn  cn 

cn 

cm 

Li_  Ll-  Li- 

Ll- 

< 

O  O  O  O  O 

X 

X 

X 

X 

X 

X 

:s 

:2  3: 

12 

O 

U-  Ll-  Ll 

Ll- 

rc  ^  IT  D=  □= 

<  c 

*< 

CL  Q-  CL 

Q- 

L. 

O 


CD 


CD 

C 

c 

o 

on 

S- 

CD 

a. 


CD 

CD 

-a 

CD 

o 


JD 

O 


c: 

O 

I 

on 

T3 

C 

03 


on 

CD 

03 

O 

on 

cjn 

c 


39 


Mean  Intercorrelations  Among  13  Summary  Criterion  Scores  for  the  Batch  A  MOS  in  the  LVI  Sample 
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ThG  number  of  soldiers  with  complete  predictor  dnd  criterion  detd  in 
each  MOS  is  reported  in  Table  1.15  for  both  the  CVI  and  LVI  data  sets. 


Table  1.15 

Soldiers  in  CVI  and  LVI  Data  Sets  With  Complete  Predictor  and  First-Tour 


Criterion 

Data  by  MOS 

MOS 

CVI 

LVI  (Listwise 

Deletion  Sample) 

IIB 

Infantryman 

491 

235 

13B 

Cannon  Crewmember 

464 

553 

19E^ 

M60  Armor  Crewman 

394 

73 

19K 

Ml  Armor  Crewman 

— 

446 

31C 

Single  Channel  Radio  Operator 

289 

172 

63B 

Light-Wheel  Vehicle  Mechanic 

478 

406 

71L 

Administrative  Specialist 

427 

252 

88M 

Motor  Transport  Operator 

507 

221 

91A 

Medical  Specialist 

392 

535 

95B 

Mi  1 itary  Police 

597 

270 

Total  4 

,039 

3,163 

MOS  19E 

not  included  in  LVI  validity  analyses. 

The 

analysis  procedure  consisted  of  the  following 

major  steps: 

A)  Using  the  listwise  deletion  sample,  multiple  correlations 
between  each  set  of  predictor  scores  and  the  five 
substantive  factor  scores,  their  five  residual  factor 
scores,  the  two  method  factor  scores,  and  the  total  scores 
from  the  hands-on  and  job  knowledge  tests  were  computed 
separately  by  MOS  and  then  averaged. 

B)  Using  the  listwise  deletion  sample,  incremental  validities 
for  each  set  of  Experimental  Battery  predictors  (e.g., 

AVOICE  composites  or  computer  composites)  over  the  four 
ASVAB  factor  composites  were  computed  against  the  same 
criteria  used  to  compute  the  validities  in  Step  A.  Once 
again,  the  results  were  computed  separately  by  MOS  and  then 
averaged. 

C)  Using  the  setwise  deletion  samples,  multiple  correlations 
and  incremental  validities  (over  the  four  ASVAB  factor 
composites)  between  each  set  of  Experimental  Battery 
predictors  and  the  criteria  used  in  the  first  two  steps  were 
computed  separately  by  MOS  and  then  averaged.  All  results 
to  this  point  were  corrected  for  range  restriction  and 
adjusted  for  shrinkage  using  the  Rozeboom  formula. 
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D)  Finally,  once  again  using  the  listwise  deletion  sample, 
multiple  correlations  and  incremental  validities  (over  the 
four  ASVAB  factors)  were  computed  for  each  set  of  predictors 
in  the  Experimental  Battery,  this  time  adjusting  the  results 
for  shrinkage  with  the  Claudy  (1978)  instead  of  the  Rozeboom 
formula.  This  step  was  conducted  to  allow  comparisons 
between  the  first-tour  validity  results  associated  with  the 
longitudinal  sample  and  those  that  had  been  reported  for  the 
concurrent  sample  (for  which  only  the  Claudy  formula  was 
used,  e.g.,  McHenry,  Hough,  Toquam,  Hanson,  &  Ashworth, 

1990). 

Multiple  Correlations  and  Incremental  Validities  Based  on  Listwise  Deletion 
Samples 

Multiple  correlations  for  the  four  ASVAB  factor  composites,  the  single 
spatial  composite,  the  eight  computer  composites,  the  three  JOB  composites, 
the  seven  ABLE  composites,  and  the  eight  AVOICE  composites  are  reported  in 
Table  1.16. 

Incremental  validity  results  for  the  Experimental  Battery  predictors 
over  the  ASVAB  factors  are  reported  in  Table  1.17.  The  results  indicate  that 
the  spatial  composite  added  slightly  to  the  prediction  of  the  raw  and  residual 
Core  Technical  and  General  Soldiering  performance  factors,  as  well  as  to  the 
written  method  factor  and  the  hands-on  and  job  knowledge  total  scores.  They 
also  show  that  the  seven  ABLE  composites  contributed  substantially  to  the 
prediction  of  the  raw  and  residual  Personal  Discipline  and  Physical  Fitness 
performance  factors. 

Multiple  Correlations  and  Incremental  Validities  Based  on  the  Setwise  Deletion 
Samples 

Multiple  correlations  for  the  spatial  composite,  the  eight  computer 
composites,  the  three  JOB  composites,  the  seven  ABLE  composites,  and  the  eight 
AVOICE  composites  based  on  the  setwise  deletion  samples  described  above  are 
reported  in  Table  1.18.  These  multiple  correlations  were  very  similar  to 
those  computed  with  the  listwise  sample.  However,  there  was  a  consistent 
difference  between  the  two  sets  of  results;  specifically,  the  multiple 
correlations  based  on  the  setwise  samples  were  generally  one  to  three  validity 
points  higher. 

Incremental  validity  results  associated  with  the  setwise  deletion 
samples  can  be  found  in  Table  1.19.  The  incremental  validity  results  based  on 
the  setwise  samples  were  practically  identical  to  those  based  on  the  listwise 
sample.  Again,  the  primary  difference  between  the  two  sets  of  results  was 
that  the  level  of  validities  was  sometimes  one  or  two  points  lower  for  the 
listwise  sample  than  for  the  setwise  samples. 


42 


Table  1.16 


Mean  of  Multiple  Correlations  Computed  Within  Job  for  LVI  Listwise  Deletion 
Samples  for  ASVAB  Factors,  Spatial,  Computer,  JOB,  ABLE  Composites,  and  AVOICE 


No.  of  Factors  Spatial  Computer  JOB  ABLE  Comp.  AVOICE 
Criterion^  MOS^  [4] _ [1] _ [0] _ [^1 _ LZJ - — 


CTP  (Raw) 

9 

62 

(13) 

57 

(11) 

47 

(16) 

29 

(13) 

21 

(09) 

38 

(08) 

GSP  (Raw) 

8 

66 

(07) 

64 

(06) 

55 

(08) 

29 

(13) 

23 

(14) 

37 

(07) 

ELS  (Raw) 

9 

37 

(12) 

32 

(08) 

29 

(15) 

18 

(14) 

13 

(11) 

17 

(15) 

MPD  (Raw) 

9 

17 

(13) 

14 

(11) 

10 

(16) 

06 

(13) 

14 

(11) 

05 

(10) 

PFB  (Raw) 

9 

16 

(06) 

10 

(04) 

07 

(07) 

06 

(06) 

27 

(07) 

05 

(09) 

CTP  (Res) 

9 

46 

(17) 

42 

(15) 

29 

(22) 

17 

(12) 

08 

(11) 

28 

(12) 

GSP  (Res) 

8 

51 

(10) 

51 

(08) 

41 

(10) 

18 

(11) 

12 

(12) 

26 

(09) 

ELS  (Res) 

9 

46 

(18) 

41 

(13) 

37 

(20) 

23 

(15) 

21 

(15) 

24 

(16) 

MPD  (Res) 

9 

18 

(13) 

14 

(12) 

08 

(16) 

07 

(11) 

13 

(11) 

06 

(10) 

PFB  (Res) 

9 

20 

(10) 

12 

(08) 

09 

(11) 

07 

(06) 

28 

(10) 

09 

(11) 

Written 

9 

54 

(13) 

49 

(12) 

43 

(18) 

29 

(16) 

23 

(12) 

29 

(14) 

Ratings 

9 

12 

(09) 

09 

(07) 

07 

(09) 

06 

(09) 

03 

(05) 

02 

(07) 

HO-Total 

9 

50 

(14) 

48 

(11) 

38 

(15) 

18 

(13) 

11 

(11) 

28 

(09) 

JK-Total 

9 

71 

(08) 

65 

(07) 

58 

(10) 

36 

(14) 

31 

(08) 

41 

(08) 

Note.  Corrected  for  range  restriction,  and  adjusted  for  shrinkage  (Rozeboo'm 
formula  8).  Numbers  in  brackets  are  the  numbers  of  predictor  scores 
entering  prediction  equations.  Numbers  in  parentheses  are  standard 
deviations.  Decimals  omitted. 

^  CTP  =  Core  Technical  Proficiency;  GSP  =  General  Soldiering  Proficiency; 

ELS  =  Effort  and  Leadership;  MPD  =  Maintaining  Personal  Discipline; 

PFB  =  Physical  Fitness  and  Military  Bearing;  HO  =  Hands-On;  JK  =  Job 

Knowledge . 

Number  of  MOS  for  which  validities  were  computed. 
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Table  1.17 


Mean  of  Incremental  Correlations  Over  ASVAB  Factors  Computed  Within  Job  for 
LVI  Listwise  Deletion  Samples  for  Spatial,  Computer,  JOB,  ABLE  Composites,  and 

AVOICE _ _ _ 


ASVAB 

Factors 

A4+ 

A4+ 

A4+ 

A4+ 

A4+ 

No.  of  (A4) 

Spatial 

Computer 

JOB 

ABLE  Comp. 

AVOICE 

MOS"  [4] 

[5] 

[12] 

[7] 

[11] 

[12] 

CTP 

(Raw) 

9 

62 

(13) 

63 

(13) 

61 

(14) 

61 

(13) 

61 

(13) 

62 

(13) 

GSP 

(Raw) 

8 

66 

(07) 

68 

(07) 

66 

(07) 

66 

(07) 

66 

(07) 

66 

(07) 

ELS 

(Raw) 

9 

37 

(12) 

36 

(13) 

35 

(13) 

36 

(13) 

34 

(17) 

33 

(16) 

MPD 

(Raw) 

9 

17 

(13) 

16 

(14) 

16 

(15) 

14 

(15) 

23 

(14) 

10 

(15) 

PFB 

(Raw) 

9 

16 

(06) 

13 

(08) 

09 

(08) 

17 

(08) 

M 

(06) 

12 

(10) 

CTP 

(Res ) 

9 

46 

(17) 

47 

(17) 

44 

(18) 

45 

(18) 

43 

(19) 

46 

(19) 

GSP 

(Res) 

8 

51 

(10) 

53 

(09) 

51 

(10) 

50 

(10) 

50 

(10) 

50 

(10) 

ELS 

MPD 

(Res) 

(Res) 

9 

9 

46 

18 

(18) 

(13) 

47 

15 

(18) 

(14) 

44 

15 

(21) 

(14) 

45 

14 

(21) 

(14) 

45 

22 

(22) 

(14) 

44 

12 

(21) 

(13) 

PFB 

(Res) 

9 

20 

(10) 

18 

(12) 

13 

(11) 

20 

(11) 

M 

(10) 

18 

(13) 

Wri 

Rat 

tten 

ings 

9 

9 

54 

12 

(13) 

(09) 

55 

11 

(13) 

(08) 

51 

09 

(18) 

(10) 

54 

09 

(13) 

(10) 

54 

09 

(12) 

(08) 

52 

05 

(17) 

(08) 

HO- 

Total 

9 

50 

(14) 

52 

(13) 

49 

(14) 

49 

(15) 

48 

(14) 

49 

(15) 

JK- 

Total 

9 

71 

(08) 

72 

(08) 

71 

(09) 

71 

(08) 

71 

(08) 

71 

(08) 

Note.  Corrected  for  range  restriction,  and  adjusted  for  shrinkage  (Rozeboom 
formula  8).  Numbers  in  brackets  are  the  numbers  of  predictor  scores 
entering  prediction  equations.  Numbers  in  parentheses  are  standard 
deviations.  Multiple  Rs  for  ASVAB  Factors  alone  are  in  italics. 
Underlined  numbers  denote  multiple  Rs  greater  than  for  ASVAB  Factors 
alone.  Decimals  omitted. 

®  Number  of  MOS  for  which  validities  were  computed. 
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Table  1.18 


Mean  of  Multiple  Correlations  Computed  Within  Job  for  LVI  Setwise  Deletion 
Samoles  for  Spatial,  Computer,  JOB,  ABLE  Composites,  and  AVOICE 

Criterion 

No.  of 
MOS^ 

Spatial 

[1] 

Computer 

[8] 

JOB 

[3] 

ABLE 

Composites 

[7] 

AVOICE 

[8] 

CTP  (Raw) 

9 

58  (11) 

49  (16) 

31  (13) 

21  (09) 

39  (07) 

GSP  (Raw) 

8 

65  (06) 

55  (08) 

32  (13) 

24  (14) 

38  (07) 

ELS  (Raw) 

9 

33  (08) 

30  (15) 

19  (14) 

12  (11) 

20  (12) 

MPD  (Raw) 

9 

14  (11) 

10  (16) 

06  (13) 

15  (11) 

05  (11) 

PFB  (Raw) 

9 

08  (04) 

13  (07) 

07  (06) 

28  (07) 

09  (09) 

CTP  (Res) 

9 

43  (15) 

31  (22) 

17  (12) 

10  (11) 

29  (09) 

GSP  (Res) 

8 

51  (08) 

40  (10) 

21  (11) 

14  (12) 

28  (09) 

ELS  (Res) 

9 

41  (13) 

36  (20) 

24  (15) 

21  (15) 

26  (06) 

MPD  (Res) 

9 

13  (12) 

10  (16) 

06  (11) 

15  (11) 

07  (13) 

PFB  (Res) 

9 

11  (08) 

10  (11) 

09  (06) 

30  (10) 

12  (10) 

Wr i tten 

9 

51  (11) 

46  (16) 

31  (17) 

25  (11) 

32  (15) 

Ratings 

9 

09  (08) 

09  (09) 

07  (08) 

04  (06) 

03  (07) 

HO-Total 

9 

50  (11) 

38  (15) 

20  (13) 

13  (11) 

30  (07) 

JK-Total 

9 

66  (07) 

60  (10) 

38  (14) 

30  (08) 

43  (08) 

Note.  Corrected  for  range  restriction  and  adjusted  for  shrinkage  (Rozeboom 
formula  8).  Numbers  in  brackets  are  the  numbers  of  predictor  scores 
entering  prediction  equations.  Numbers  in  parentheses  are  standard 
deviations.  Decimals  omitted. 

^  Number  of  MOS  for  which  validities  were  computed. 
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Table  1.19 


Mean  of  Incremental  Correlations  Over  ASVAB  Factors  Computed  Within  Job  for 
LVI  Setwise  Deletion  Samples  for  Spatial,  Computer,  JOB,  ABLE  Composites,  and 
AVOICE  _ _ _ _ _ 


Criterion 

No.  of 
MOS" 

ASVAB 

Factors  (A4) 

+  Spatial 
[5] 

A4+ 

Computer 

[12] 

A4+ 

JOB 

[7] 

A4+  ABLE 
Composites 
[11] 

A4+ 

AVOICE 

[12] 

CTP  (Raw) 

9 

64  (10) 

61  (11) 

63  (11) 

61  (12) 

64  (11) 

GSP  (Raw) 

8 

69  (06) 

66  (07) 

67  (07) 

66  (08) 

66  (07) 

ELS  (Raw) 

9 

37  (10) 

36  (14) 

37  (11) 

36  (13) 

36  (11) 

MPD  (Raw) 

9 

15  (13) 

15  (15) 

12  (13) 

24  (13) 

11  (14) 

PFB  (Raw) 

9 

15  (08) 

17  (05) 

IZ  (07) 

32  (04) 

15  (10) 

CTP  (Res) 

9 

48  (12) 

45  (14) 

46  (14) 

45  (14) 

47  (14) 

GSP  (Res) 

8 

54  (06) 

50  (08) 

51  (08) 

50  (07) 

50  (07) 

ELS  (Res) 

9 

47  (12) 

43  (20) 

46  (15) 

46  (15) 

46  (14) 

MPD  (Res) 

9 

14  (13) 

13  (15) 

13  (13) 

22  (12) 

11  (14) 

PFB  (Res) 

9 

20  (11) 

18  (11) 

20  (10) 

36  (08) 

21  (11) 

Written 

9 

57  (13) 

53  (17) 

58  (12) 

55  (13) 

54  (18) 

Ratings 

9 

10  (09) 

n  (11) 

11  (09) 

11  (07) 

06  (09) 

HO-Total 

9 

53  (09) 

49  (11) 

50  (12) 

49  (11) 

50  (11) 

JK-Total 

9 

73  (08) 

71  (09) 

72  (08) 

71  (09) 

71  (09) 

Note.  Corrected  for  range  restriction  and  adjusted  for  shrinkage  (Rozeboom 
formula  8).  Numbers  in  brackets  are  the  numbers  of  predictor  scores 
entering  prediction  equations.  Numbers  in  parentheses  are  standard 
deviations.  Underlined  numbers  denote  multiple  Rs  greater  than  for 
ASVAB  Factors  alone.  Decimals  omitted. 

^  Number  of  MOS  for  which  validities  were  computed. 
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Comparison  of  Validity  Research  in  LVI  and  CVI  Samples 

The  final  set  of  results  concern  the  comparison  between  the  validity 
estimates  associated  with  the  longitudinal  data  (i.e.,  LVI)  and  those  reported 
for  the  concurrent  validation  data  (CVI).  Table  1.20  reports  the  multiple 
correlations  for  the  ASVAB  factors  and  each  set  of  experimental  predictors  as 
computed  for  the  listwise  sample  in  both  data  sets. 

The  results  in  Table  1.20  demonstrate  that  the  patterns  and  levels  of 
validities  are  very  similar  across  the  two  sets  of  analyses.  Still,  there  are 
some  differences  worth  pointing  out.  Specifically,  in  comparison  to  the 
results  of  the  CVI  analyses:  (a)  The  LVI  validities  of  the  "cognitive" 
predictors  (i.e.,  ASVAB,  spatial,  computer)  for  predicting  the  "wi  1  do 
performance  factors  (ELS,  MPD,  and  PFB)  are  higher;  (b)  the  LVI  validities  of 
the  ABLE  composites  for  predicting  the  "will  do"  performance  factors  are 
somewhat  lower;  and  (c)  the  LVI  validities  of  the  AVOICE  composites  for 
predicting  the  "can  do"  performance  factors  (CTP  and  GSP)  are  higher. 


Table  1.20 

Comparison  of  Mean  Multiple  Correlations  Computed  Within  Job  for  LVI  and  CVI 
Listwise  Deletion  Samples  for  ASVAB  Factors,  Spatial,  Computer,  JOB,  ABLE 
Composites,  and  AVOICE  _ _ _ 


ASVAB 

Factors 


Spatial  Computer 


ABLE 

Comp.  AVOICE 
LV  CV  LV  CV  LV  CV 


JOB 


Criterion 

nu  •  1 

MOS® 

l_  V 

[4] 

[4] 

[1] 

[1] 

[8] 

[6] 

[3] 

[3] 

[7] 

[4] 

[8] 

[6] 

CTP  (Raw) 

9 

63 

63 

57 

56 

50 

53 

31 

29 

27 

26 

41 

35 

GSP  (Raw) 

8 

67 

65 

64 

63 

57 

57 

32 

30 

29 

25 

40 

34 

ELS  (Raw) 

9 

39 

31 

32 

25 

34 

26 

22 

19 

20 

33 

25 

24 

MPD  (Raw) 

9 

22 

16 

14 

12 

15 

12 

11 

11 

22 

32 

11 

13 

PFB  (Raw) 

9 

21 

20 

10 

10 

17 

11 

12 

11 

31 

37 

15 

12 

CTP  (Res) 

9 

48 

47 

42 

37 

35 

37 

20 

21 

18 

22 

33 

28 

GSP  (Res) 

8 

53 

49 

51 

48 

44 

41 

22 

22 

19 

21 

31 

26 

ELS  (Res) 

9 

48 

46 

41 

41 

40 

38 

25 

27 

26 

31 

29 

32 

MPD  (Res) 

9 

23 

19 

14 

15 

14 

13 

12 

10 

21 

28 

13 

15 

PFB  (Res) 

9 

24 

21 

12 

11 

17 

14 

11 

10 

32 

35 

16 

14 

Written 

9 

56 

62 

49 

55 

47 

54 

31 

28 

29 

21 

33 

32 

Ratings 

9 

16 

15 

09 

07 

17 

08 

10 

08 

09 

18 

09 

09 

Note.  Corrected  for  range  restriction  and  adjusted  for  shrinkage  (Claudy 
formula).  Numbers  in  brackets  are  the  numbers  of  predictor  scores 
entering  prediction  equations.  Decimals  omitted. 

®  Number  of  MOS  for  which  validities  were  computed. 
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Further  Exploration  of  ELS  and  ABLE 


As  shown  in  the  date  reported  above,  the  largest  difference  between  the  CVI 
and  LVI  validation  results  was  in  the  prediction  of  the  Effort  and  Leadership 
(ELS)  performance  factors  with  the  ABLE  basic  scores.  Corrected  for  restriction 
of  range  and  for  shrinkage,  the  validity  of  the  four  ABLE  composite  scores  in  CVI 
was  .33  for  ELS  and  the  validity  of  the  seven  ABLE  factor  scores  in  LVI  was  .20. 
Whencast  against  the  variability  in  results  across  studies  in  the  extant 
literature,  such  a  difference  may  not  seem  all  that  large  or  very  unusual. 

However,  since  the  obtained  results  from  CVI,  CVII,  and  LVI  have  been  so 
consistent,  in  terms  of  the  expected  convergent  and  divergent  results,  we 
subjected  this  particular  difference  to  a  series  of  additional  analyses  in  an 
attempt  to  determine  the  reason  for  the  discrepancy. 

First,  the  discrepancy  does  not  seem  to  arise  from  any  general 
deterioration  in  the  measurement  properties  of  either  the  ABLE  or  the  ELS 
composite  in  the  LVI  sample.  For  example,  while  the  correlation  of  the  ABLE  with 
ELS  and  MPD  went  down,  the  ABLE's  correlations  with  CTP  and  6SP  went  up  slightly. 
Similarly  a  decrease  in  the  validity  with  which  ELS  was  predicted  was 
characteristic  only  of  the  ABLE.  The  validities  of  the  cognitive  measures,  the 
JOB,  and  AVOICE  for  predicting  ELS  actually  increased  by  varying  amounts. 
Consequently,  the  decrease  in  validity  seems  to  be  specific  to  the  ABLE/ELS 
correlation  and,  to  a  lesser  extent,  the  ABLE/MPD  correlation. 

The  followup  analyses  were  also  able  to  rule  out  two  possible  additional 
sources  of  the  CVI/LVI  validity  differences.  First,  differences  in  the 
composition  and  number  of  ABLE  basic  scores  from  CVI  to  LVI  did  not  account  for 
the  differences  in  patterns  of  validity.  Second,  differences  in  the  composition 
of  the  Effort/Leadership  factor  score  from  CVI  to  LVI  did  not  account  for 
differences  in  validity. 

Rather,  the  somewhat  lower  correlation  of  ABLE  with  Effort/Leadership  in 
LVI  seems  due  to  the  joint  effects  of  two  influences,  the  determinants  of 

ELS  scores  seem  to  favor  ability  slightly  more  and  motivation  slightly  less  in 
LVI  versus  CVI,  perhaps  because  their  true  score  variances  were  different  across 
the  two  cohorts.  Second,  the  greater  influence  of  the  social  desirability 
response  tendency  in  LVI  seems  to  produce  more  positive  manifold  (i.e.,  higher 
intercorrelations  for  the  LVI  ABLE  basic  scores),  as  contrasted  with  CVI. 
could  also  lower  the  correlation  of  the  regression-weighted  ABLE  composite  with 
ELS,  whereas  it  might  not  have  the  same  effect  with  the  Core  Technical  and 
General  Soldiering  factors. 

Yet  another  component  of  the  explanation  is  the  negative  correlation 
between  the  Social  Desirability  scale  and  AFQT.  AFQT  and  Social  Desirability 
correlated  -.22  in  the  CV  sample  and  -.20  in  the  LV  sample.  This  would  tend  to 
lower  the  correlation  between  ABLE  and  ELS  if  the  correlations  between  ABLE  and 
ASVAB  and  between  ASVAB  and  ELS  were  positive,  which  they  were. 

Summary  of  LVI  Validation 

Generally  speaking,  the  ASVAB  was  the  best  predictor  of  performance. 
However,  the  composite  of  spatial  tests  provided  a  small  amount  of  incremental 
validity  for  the  "can  do"  criteria  (1-3  points),  and  the  ABLE  provided  larger 
increments  (7-20  points)  for  two  of  the  three  "will  do"  criteria  (Maintaining 
Personal  Discipline,  and  Physical  Fitness  and  Bearing).  Estimates  of  incremental 
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validity  were  somewhat  higher  when  the  results  were  not  corrected  for  range 
restriction. 

With  regard  to  ASVAB  scoring  options,  results  indicate  a  very  slight  edge 
for  using  multiple  regression  equations  based  on  the  four  ASVAB  unit-  weighted 
factor  scores.  In  the  test  of  ABLE  scoring  options,  the  method  using  factor 
scores  computed  from  a  subset  of  all  the  ABLE  items  (ABLE-114)  proved  to  have 
consistently  slightly  higher  validities. 

Perhaps  the  most  interesting  finding  is  derived  from  the  comparisons 
between  the  Longitudinal  Validation  results  and  those  from  the  Concurrent 
Validation.  Generally  speaking,  the  pattern  and  level  of  the  validity 
coefficients  were  highly  similar  across  the  two  samples.  The  correlation  between 
the  CV  and  LV  coefficients  in  Table  1.20  was  .962  and  the  root  mean-squared 
difference  between  the  two  sets  of  coefficients  was  .046.  However,  the 
correlation  is  not  1.00.  As  noted  above,  the  longitudinal  validities  were  higher 
for  cognitive  predictors  against  "will  do"  criteria  and  lower  for  ABLE  composites 
against  "will  do"  criteria.  Some  of  the  possible  explanations  for  those 
differences  include  changes  in  the  nature  of  predictor  scores  when  administered 
in  a  longitudinal  versus  concurrent  design,  changes  in  criterion  or  predictor 
scores  due  to  cohort  differences,  and  changes  in  the  true  relationship  between 
abilities  and  performance  as  persons  gain  more  experience  and  training  in  an 
organization  and  job.  These  and  other  possible  explanations  will  be  explored  in 
future  analyses. 

Results  of  the  Concurrent  Sample  Second-Tour  Validation  (CVII) 

The  CVII  validation  results  are  based  on  the  CVII  sample,  which  was 
assessed  on  the  criterion  measures  of  second-tour  performance  at  the  same  time 
that  the  LVI  performance  data  were  collected  from  the  fjrst-tour  longitudinal 
sample.  The  predictor  set  is  limited  to  ASVAB  and  ABLE  because  only  a  small 
proportion  (approximately  12%)  of  the  CVII  sample  had  been  assessed  with  the 
Experimental  Predictor  Battery.  ASVAB  scores,  taken  5-6  years  earlier,  were 
available  from  the  Enlisted  Master  File.  The  ABLE  was  administered  concurrently 
during  the  CVII  data  collection  to  approximately  45  percent  of  the  total  sample 
(i.e.,  those  individuals  who  had  no  peers  in  the  sample  to  rate  and  thus  had  time 
to  take  the  ABLE).  Everyone  in  the  sample  was  assessed  on  the  full  set  of 
second-tour  performance  measures.  By  design,  the  MOS  in  the  CVII  sample  were 
limited  to  the  MOS  in  Batch  A.  Because  of  the  generally  small  samples  for 
individual  MOS,  results  for  most  analyses  are  reported  for  the  combined  sample. 

The  CVII  data  collection  and  data  presentation  are  described  in  the  first 
annual  report  for  Building  the  Career  Force  (Campbell  &  Zook,  1990;  see  Chapters 
5  and  6).  After  final  editing,  the  total  N  for  CVII  was  1,053.  The  total  sample 
was  distributed  across  the  Batch  A  MOS  as  shown  in  Table  1.21. 

Because  of  some  missing  data,  the  sample  sizes  varied  depending  on  the 
specific  analysis  being  reported.  For  example,  for  the  reasons  cited  above,  ABLE 
scores  were  available  for  only  477  individuals.  All  the  analyses  that  require  a 
common  covariance  matrix  for  ABLE  and  ASVAB  were  based  on  this  reduced  sample. 

The  development  of  the  CVII  performance  measures,  and  the  analysis  and 
modeling  of  CVII  performance,  all  have  been  described  previously  (Campbell  & 

Zook,  1990)  and  are  summarized  in  a  previous  section  of  the  present  chapter.  The 
solution  that  yielded  the  best  fit  consisted  of  six  substantive  factors  and  two 
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Table  1.21 


evil  Sample  Sizes  by  MOS _ 

MOS  _ N 


IIB 

Infantryman 

127 

13B 

Cannon  Crewmember 

162 

19E 

M60  Armor  Crewman 

33 

19K 

Ml  Armor  Crewman 

10 

31C 

Single  Channel  Radio  Operator 

103 

63B 

Light-Wheel  Vehicle  Mechanic 

116 

71L 

Administrative  Specialist 

112 

88M 

Motor  Transport  Operator 

144 

91A 

Medical  Speciali'st 

146 

95B 

Military  Police 

141 

Total 

1,053 

methods  factors.  The  two  methods  factors  were  defined  to  be  orthogonal  to  the 
substantive  factors,  but  the  correlations  among  the  substantive  factors  were  not 
so  constrained.  The  six  substantive  factors  and  two  methods  factors,  and  the 
variables  that  are  scored  on  each,  were  shown  in  Figure  1.9. 


The  complete  basic  validation  analyses  utilized  a  total  of  10  scores  for 
the  performance  factors,  as  shown  below. 


(1)  (2)  (3) 

TCS  CTP  GSP 


(7)  (CTP  +  GSP) 


(4)  (5)  (6) 

ELS  MPD  PFB 


(8)  (ELS  +  MPD) 


(9)  (TCS  +  C 


P  +  GSP) 


(10)  (ELS  +  MPD  +  PFB) 


TCS  =  Training/Counseling  Subordinates;  CTP  =  Core  Technical  Proficiency;  GSP  =  General 
Soldiering  Proficiency;  ELS  =  Effort/Leadership;  MPD  =  Maintaining  Personal  Discipline; 
PFB  =  Physical  Fitness  and  Bearing 


That  is,  all  10  scores  were  used  as  criterion  measures.  All  higher  order 
composite  scores  were  obtained  by  standardizing  the  component  scores  and  then 
taking  the  simple  sum. 
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Procedure 


The  evil  validation  analysis  procedure  consisted  of  the  following  steps. 

(1)  The  ASVAB  and  ABLE  were  correlated  with  the  six  performance 
factor  scores,  their  five  residual  scores  (there  was  no 
residual  for  TCS),  the  higher  order  factor  composites,  the  two 
methods  factor  scores,  and  the  total  score  from  the  hands-on 
tests,  the  job  knowledge  tests,  and  the  Situational  Judgment 
Test.  ASVAB  was  represented  by  the  AFQT,  a  regression-weighted 
composite  of  the  four  factors,  and  a  regression-weighted 
composite  of  the  nine  subtests.  ABLE  was  represented  by  the 
three  alternative  sets  of  scores  described  previously.  Both 
corrected  (for  multivariate  restriction  of  range)  and 
uncorrected  estimates  were  computed,  and  both  regression 
weights  and  unit  weights  (applied  to  standardized  scores)  were 
used.  When  multiple  regression  weights  were  used,  the  Rozeboom 
correction  (Rozeboom,  1978)  was  used  to  account  for  the  fitting 
of  error. 

(2)  As  in  CVI,  incremental  validities  for  the  ABLE  composites  over 
the  ASVAB  composites  were  also  computed  against  each  criterion 
score. 

(3)  A  hierarchical  regression  analysis,  stopping  at  six  predictors, 
was  run  against  each  performance  factor,  factor  composite,  and 
individual  criterion  score  (i.e.,  hands-on,  job  knowledge,  and 
Situational  Judgment  Test). 

(4)  A  hierarchical  regression  analysis  was  also  carried  out  on 
selected  criterion  variables  for  the  combined  samples  from 
three  MOS  clusters.  The  clusters  were  based  on  the  results  of 
an  MOS  clustering  within  the  Synthetic  Validation  Project 
(Wise,  Peterson,  Hoffman,  Campbell,  &  Arabian,  1991)  and  on  the 
results  of  the  validity  generalization  analysis  for  the  Batch  A 
MOS  in  the  CVI  sample  (Wise,  McHenry,  &  Campbell,  1990). 

(5)  The  final  step  consisted  of  using  the  optimal  six  variable 
equations  from  the  hierarchical  regression  analyses  described 
above  to  develop  a  picture  of  the  {Jegree  of  differential 
prediction  across  performance  factors  and  across  the  three  MOS 
clusters. 

Results 


The  basic  multiple  correlations  for  ASVAB  (four  factors  vs.  nine 
subtests)  and  ABLE  (seven  theoretically  based  composites  vs.  seven  "purified" 
empirical  factors)  are  given  in  Table  1.22.  Several  things  are  worth  noting. 
ASVAB,  taken  at  time  of  entry,  is  still  a  highly  valid  predictor  of  Core 
Technical  and  General  Soldiering  Proficiency  and  has  respectable  validity  for 
Effort/Leadership.  For  ASVAB,  the  four  factors  and  the  nine  subtests  provide 
virtually  the  same  level  of  predictive  accuracy.  However,  for  ABLE  the 
reduced  factor  scores  (114  items)  are  consistently  the  best  predictor  set. 
ABLE  predicts  Effort/Leadership  and  Physical  Fitness  very  well  and  has 
reasonable  correlations  with  General  Soldiering  and  Training/Counseling. 
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Table  1.22 


Multiple  Correlations  for  ASVAB  Factors,  ASVAB  Subtests,  ABLE  Composites,  and 
ABLE-114  Scores  Against  19  CVII  Criterion  Variables  (All  MOS),  With  Unit 
Weights _ _ _ _ _ _ _ 


ASVAB  ASVAB  ABLE 

Variable  Factors  Subtests  Composites  ABLE-114 

[4] _ [9] _ [7] _ [7] 


Core  Technical  (Raw)  43 
General  Soldiering  (Raw)  56 
Effort/Leadership  (Raw)  38 
Personal  Discipline  (Raw)  00 
Physical  Fitness  (Raw)  13 
Training/Counseling  (Raw)  06 

Core  Technical  (Res)  29 
General  Soldiering  (Res)  42 
Effort/Leadership  (Res)  25 
Personal  Discipline  (Res)  00 
Physical  Fitness  (Res)  16 

ELS  -  No  Situational  Judgment  24 
Criterion  Composite  CTP/GSP  57 
Criterion  Composite  ELS/MPD  29 
Criterion  Factor  1  CTP+GSP+TCS  50 
Criterion  Factor  2  ELS+MPD+PFB  14 
Hands-On  Average  39 
Job  Knowledge  Total  59 
Situational  Judgment  42 


(42) 

43 

(43) 

15 

(14) 

20 

(15) 

(54) 

57 

(55) 

22 

(16) 

26 

(18) 

(38) 

39 

(38) 

37 

(32) 

41 

(32) 

(11) 

00 

(11) 

20 

(21) 

18 

(22) 

(16) 

06 

(16) 

32 

(23) 

34 

(21) 

(13) 

00 

(12) 

27 

(19) 

23 

(18) 

(29) 

28 

(30) 

00 

(12) 

07 

(13) 

(42) 

43 

(42) 

14 

(15) 

18 

(16) 

(26) 

27 

(25) 

38 

(31) 

41 

(30) 

(09) 

00 

(09) 

16 

(20) 

15 

(19) 

(20) 

09 

(20) 

34. 

(21) 

35 

(18) 

(22) 

23 

(22) 

34 

(31) 

38 

(30) 

(55) 

58 

(56) 

22 

(17) 

27 

(19) 

(30) 

29 

(29) 

34 

(32) 

37 

(32) 

(50) 

50 

(50) 

29 

(22) 

32 

(23) 

(16) 

12 

(15) 

34 

(35) 

35 

(34) 

(40) 

38 

(40) 

12 

(12) 

18 

(13) 

(56) 

59 

(57) 

25 

(14) 

28 

(16) 

(43) 

42 

(43) 

27 

(20) 

31 

(21) 

Note.  N  =  412.  Adjusted  (Rozeboom  formula).  Numbers  in  brackets  are  the 

number  of  predictor  scores  entering  prediction  equations.  Validities  of 
unit-weighted  composites  are  in  parentheses.  Decimals  omitted. 


In  general,  after  adjustments,  regression  weights  and  unit  weights  for 
ASVAB  yield  about  the  same  level  of  validity.  However,  regression  weights  are 
somewhat  better  than  unit  weights  for  the  seven  empirical  ABLE  factors.  There 
is  not  as  much  positive  manifold  among  the  ABLE  factors  as  there  is  among  the 
ASVAB  subtests. 

Table  1.23  contains  the  same  type  of  incremental  analyses  that  were  done 
in  CVI  (Campbell  &  Zook,  1991).  ABLE  does  not  add  to  the  prediction  of  Core 
Technical  and  General  Soldiering  Proficiency,  but  it  adds  about  the  same 
amount  to  the  prediction  of  Effort/Leadership  as  it  did  in  CVI.  However, 
the  overall  level  of  prediction  for  ELS  is  higher  in  CVII  than  it  was  in 
CVI  (R  =  .50  vs.  .43). 
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Table  1.23 


Multiple  Correlations  for  ASVAB  Factors  Plus  ABLE  Composites  and  Plus  ABLE-114 
Scores,  and  for  ASVAB  Subtests  Plus  ABLE  Composites  and  Plus  ABLE-114  Scores 
Against  19  CVII  Criterion  Variables,  All  MOS  _ _ _ 


Variable 

4  ASVAB 
Factors  + 

7  ABLE  Comp 
(K=ll) 

4  ASVAB 
Factors  + 

7  ABLE-114 
(K=ll) 

9  ASVAB 
Subtests  + 

7  ABLE  Comp 
(K=16) 

9  ASVAB 
Subtests  + 

7  ABLE-114 
(K=16) 

Core  Technical  (Raw) 

.42 

.43 

.42 

.43 

General  Soldiering  (Raw) 

.56 

•  57 

.  58 

.  58 

Effort/Leadership  (Raw) 

.49 

.49 

.49 

.  50 

Personal  Discipline  (Raw) 

.16 

.13 

.09 

.03 

Physical  Fitness  (Raw) 

.34 

.35 

.32 

.33 

Training/Counseling  (Raw) 

.26 

.20 

.24 

.17 

Core  Technical  (Res) 

.24 

.26 

.24 

.25 

General  Soldiering  (Res) 

.42 

.42 

.44 

.44 

Effort/Leadership  (Res) 

.43 

.43 

.43 

.43 

Personal  Discipline  (Res) 

.09 

.07 

.00 

.00 

Physical  Fitness  (Res) 

.36 

.37 

.34 

•  34 

ELS  -  No  Situational  Judgment 

.39 

.41 

.38 

.41 

Criterion  Composite  CTP/GSP 

.57 

.57 

.  58 

.  58 

Criterion  Composite  ELS/MPD 

.40 

.40 

.40 

.40 

Criterion  Factor  1:  CTP+GSP+TCS 

.54 

.54 

.  54 

.  54 

Criterion  Factor  2:  ELS+MPD+PFB 

.35 

.35 

.34 

.34 

Hands-On  Average 

.37 

.37 

.  37 

.37 

Job  Knowledge  Total 

.60 

.60 

.60 

.60 

Situational  Judgment 

.45 

.44 

.45 

.44 

Note.  N  =  412.  Corrected  for  range  restriction  and  adjusted  (Rozeboom 
formula). 


The  hierarchical  procedure  asked  for  the  optimal  six-variable  equation. 
For  any  specific  criterion  measure  the  first  four  variables  were  never  all 
from  ASVAB  or  all  from  ABLE.  It  appears  that  ABLE,  most  frequently  the 
Dependability  scale,  does  play  a  role  in  predicting  CTP  and  GSP.  This 
contribution  is  masked  when  the  non-hierarchical  procedure  is  used. 

Generalizabilitv 

A  descriptive  picture  of  the  generalizability  of  prediction  equations 
across  performance  factors  (for  the  combined  sample)  is  shown  in  Table  1.24. 
All  entries  are  multiple  correlations  and  the  diagonals  represent  estimates 
based  on  optimal  weights.  Estimates  of  what  happens  when  less  than  optimal 
weights  are  used  to  predict  the  same  criterion  are  obtained  by  looking  across 
the  rows.  Estimates  of  what  happens  when  a  particular  set  of  weights  is 
applied  to  other  criterion  measures  or  other  MOS  are  obtained  by  looking 
down  the  columns.  All  estimates  are  based  on  the  corrected  covariance  matrix. 
The  diagonals  are  adjusted  for  shrinkage  using  the  Rozeboom  formula  with 
k  =  6.  The  off-diagonals  are  not  adjusted  because  the  weights  were  not 
computed  against  that  particular  dependent  variable. 


53 


Multiple  Correlations  for  10  Sets  of  Criterion  Composite  Weights,  A11  MOS 
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iote.  Rows  are  criteria;  columns  are  weights  corrected  for  range  restriction;  multiple  R  for  optimal  weights  in  bold;  Rozeboom  adjustments 
in  parentheses. 

Criterion  Factor  1  =  CTP+GSP+TCS. 

'  Criterion  Factor  2  =  ELS+MPD+PFB. 


As  shown  in  Tdbio  1.24,  within  MOS  thers  is  vory  little  differential 
validity  for  Core  Technical  vs.  General  Soldiering  Proficiency.  Either  set  of 
weights  works  about  as  well.  However,  the  same  is  not  the  case  for  the  other 
four  performance  factors.  Better  prediction  is  always  achieved  by  using  the 
equation  developed  for  each  factor. 

Summary  of  LVII  Validity  Estimates 

In  general,  in  spite  of  the  small  samples  for  each  MOS  and  the 
necessity  of  regarding  all  mean  criterion  differences  as  error  (i.e., 
standardizing  criterion  scores  within  MOS),  the  validities  for  ASVAB  and  ABLE 
were  as  high,  or  higher,  for  predicting  second-tour  performance  as  for 
predicting  first-tour  performance.  While  unit  weights  did  not  weaken  the 
validities  for  ASVAB,  they  did  constrain  the  predictive  accuracy  of  ABLE. 

A  consistent  finding  from  the  hierarchical  analysis  is  that  for  the 
Core  Technical  Proficiency,  General  Soldiering  Proficiency,  and  Effort/ 
Leadership  criteria,  the  optimal  predictor  battery  is  never  composed  of  only 
ASVAB  or  only  ABLE  factor  scores.  For  example,  the  Dependability  factor  from 
the  ABLE  is  a  consistent  predictor  of  the  "can  do"  component  of  performance. 

Finally,  based  on  the  above  analyses,  there  appears  to  be  more 
differential  validity  across  MOS  for  the  second-tour  samples  than  was  found 
during  the  analyses  of  the  first-tour  data  in  CVI. 

All  of  these  issues  can  be  analyzed  more  rigorously  when  the  larger 
samples  and  fuller  set  of  predictor  measures  from  the  second-tour  longitudinal 
(LVII)  validation  are  analyzed. 

Prediction  of  Second-Tour  Performance  From  the  Trial  Battery  and 

From  First-Tour  Performance 

The  fundamental  research  designs  for  Project  A  and  Career  Force  include 
the  concept  of  combining  successive  pieces  of  information  from  (a)  predictor 
tests  administered  at  entry,  (b)  measures  of  performance  during  training,  and 
(c)  measures  of  first-tour  job  performance  to  predict  individual  performance 
in  the  second  tour  of  duty.  Pending  fuller  analyses  as  larger  samples  become 
available  from  later  data  collections,  a  preliminary  explanation  was  conducted 
with  the  small  samples  from  CVII  available  at  this  stage. 

These  analyses  of  CVI  and  CVII  data  examine  the  relationship  of  ASVAB 
scores  (from  tests  given  at  the  time  recruits  entered  the  Army),  the  CVI 
predictor  scores  (i.e.,  the  Project  A  CVI  Trial  Battery,  the  preliminary 
version  of  the  Experimental  Predictor  Battery,  given  during  the  first  tour), 
and  first-tour  job  performance  scores  to  second-tour  CVII  job  performance 
scores.  Two  complications  with  these  initial  analyses  were  that  available 
sample  sizes  were  extremely  small,  and  it  was  unclear  exactly  how  to  account 
for  range  restriction  for  a  sample  of  this  type. 

There  were  121  soldiers  in  Batch  A  MOS  who  had  been  assessed  on  at 
least  a  subset  of  measures  during  the  CVI  and  CVII  data  collections.  Not  all 
121  soldiers  had  complete  CVI  and  CVII  data;  the  lowest  number  available  for  a 
given  combination  of  CVI  and  CVII  measures  was  102.  Table  1.25  shows  the 
number  of  soldiers  who  had  CVI  and  CVII  data,  by  MOS. 
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Table  1.25 


Numbers  of  Soldiers  With  CVI  and  CVII  Data  by  MOS: 

Initial  Sample 

MOS 

N 

IIB 

Infantryman 

8 

I3B 

Cannon  Crewmember 

26 

I9E 

M60  Armor  Crewman 

4 

SIC 

Single  Channel  Radio  Operator 

8 

63B 

Light-Wheel  Vehicle  Mechanic 

25 

7IL 

Administrative  Specialist 

15 

88M 

Motor  Transport  Operator 

7 

9IA/B 

Medical  Specialist 

15 

95A 

Mi  1 itary  Pol  ice 

13 

Total 

121 

Measures 


The  second-tour  performance  criterion  CVII  measures  used  in  the 
exploratory  analysis  were  the  raw  and  residual  scores  for  the  five  constructs 
first  identified  during  the  first-tour  Concurrent  Validation,  and  confirmed  by 
the  CVII  modeling  analysis.  Predictor  measures  came  from  the  ASVAB,  from  the 
Project  A  CVI  Trial  Battery,  and  from  first-tour  job  performance  measures. 
Because  of  the  extremely  limited  sample  sizes,  the  least-squares  weights 
developed  for  the  CVI  criterion  constructs  were  used  rather  than  developing 
new  weights  for  CVII  criterion  constructs. 

Analysis  and  Results 

CVT  predictor  scores  were  correlated  with  the  CVII  criterion  scores  in 
two  ways:  (a)  Correlations  were  computed  within  each  MOS  and  these  values 
were  averaged  (weighted  by  N),  and  (b)  correlations  were  computed  across  the 
total  sample.  Correlations  with  CVII  criteria  were  computed  separately  for 
the  ASVAB,  spatial,  computer-administered,  ABLE,  AVOICE,  and  JOB  composites 
and  for  the  CVI  criterion  scores.  Correlations  were  also  computed  for  the 
ASVAB  plus  each  of  the  other  predictor  sets  from  the  Trial  Battery  and  the  CVI 
criteria.  When  the  CVI  criteria  were  combined  with  any  of  the  other  predictor 
scores,  they  were  standardized  within  MOS  (using  the  larger  CVI  samples  to 
compute  standard  scores)  and  summed  to  achieve  equal  weighting  between 
ASVAB/Trial  Battery  and  CVI  criterion  scores. 

Because  of  the  number  of  different  points  at  which  additional  range 
restriction  could  occur,  there  are  a  number  of  different  "populations"  to 
which  the  CVII  sample  could  be  corrected.  If  the  problem  is  to  select  second- 
tour  soldiers  from  experienced  first-tour  personnel,  then  the  set  of  all 
persons  who  are  nearing  completion  of  the  first  tour  seems  the  most 
appropriate  population. 
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The  correlations  of  scores  on  the  first-tour  criteria  with  scores  on 
second-tour  criteria  in  the  combined  sample  are  shown  in  Table  1.26.  The 
correlations  are  not  corrected  for  restriction  of  range.  The  note  for  the 
table  shows  the  mean  of  the  diagonal  correlations,  which  contains  the 
correlations  of  the  same  criteria  across  first  and  second  tour--that  is,  the 
correlation  of  Core  Technical  between  first  and  second  tour,  and  so  on.  This 
mean  is  an  index  of  convergent  validity  for  the  set  of  criterion  constructs. 
The  note  also  shows  the  mean  of  the  off-diagonal  correlations--that  is,  the 
correlations  between  different  criterion  constructs  across  first  and  second 
tour.  The  difference  between  the  mean  diagonal  and  mean  off-diagonal 
correlation  can  be  thought  of  as  an  indicator  of  discriminant  validity. 


Table  1.26 


Uncorrected  Correlations  Between  CVI 
Computed  Across  MOS:  Initial  Sample 

and  CVII  Raw  Criterion  Composites 

CVII 

Criterion 

Composite 

CVI  Criterion 
Composite 

CTP 

GSP 

ELS 

MPD 

PFB 

Core  Technical 
Proficiency 

.47 

.48 

.22 

.10 

.08 

General  Soldiering 
Proficiency 

.47 

.43 

.36 

.13 

.17 

Effort  and 

Leadership 

.19 

.07 

.30 

.19 

.13 

Maintaining  Personal 
Discipline 

.06 

.14 

.16 

.26 

.19 

Physical  Fitness  and 
Military  Bearing 

.00 

-.04 

.15 

.15 

.48 

Note.  Ns  =  102-121. 

Mean  diagonal  value  = 

.39;  mean 

off-diagonal  value 

=  .17. 

The  correlations  of  predicted  scores  based  on  CVI  weights  for  ASVAB  and 
Trial  Battery  composites  and  CVI  criterion  scores  with  CVII  criteria  are  shown 
in  Table  1.27.  On  the  whole,  of  all  the  predictors,  the  CVI  criterion  scores 
have  the  highest  correlations  with  CVII  criterion  scores.  However,  adding  the 
ASVAB  and  the  ASVAB  plus  Trial  Battery  composite  scores  to  CVI  scores  does 
increment  the  CVI  validity  coefficients. 

The  ASVAB  validities  follow  the  familiar  pattern  of  predicting  the  two 
"can  do"  criteria,  but  not  predicting  the  "will  do"  criteria  very  well.  The 
JOB  unexpectedly  did  the  best  job  of  predicting  Maintaining  Personal 
Discipline. 

In  sum,  the  results  with  this  small  initial  sample  provide  evidence  that 
ASVAB  scores,  weighted  on  the  basis  of  regression  estimates  for  predicting 
first-tour  performance,  predict  second-tour  "can  do"  performance  with 
substantial  validity.  The  results  also  provide  evidence  of  convergent  and 
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Table  1.27 

Correlations  Between  CVI  Weighted  Predictor  Composites,  CVI  Criterion 
Composites,  and  CVII  Criterion  Composites  for  Raw  Scores,  Computed  Across  MOS: 


Initial  Sample 


Predictor  and  CVI 

Criterion  Composites 
and  Combinations 

CVII 

Criterion 

Composite 

CTP 

GSP 

ELS 

MPD 

PFB 

ASVAB 

.33 

.42 

.11 

-.05 

.11 

CVI  Performance 

.47 

.43 

.30 

.26 

.48 

ASVAB+CVI  Performance 

.51 

.51 

.33 

.26 

.47 

Computer  Tests 

.23 

.13 

-.01 

-.04 

.10 

ASVAB+Computer  Tests 

.37 

.41 

.13 

.05 

.  12 

ASVAB+Comp.  Tests+CVI  Performance 

.52 

.51 

.33 

.27 

.46 

AVOICE 

.15 

.16 

.06 

-.02 

.06 

ASVAB+AVOICE 

.43 

.44 

.14 

.00 

.  13 

ASVAB+AVOICE+CVI  Performance 

.54 

.52 

.33 

.27 

.  46 

JOB 

.12 

.00 

.19 

.30 

.12 

ASVAB+JOB 

.33 

.41 

.16 

.20 

.16 

ASVAB+JOB+CVI  Performance 

.51 

.51 

.34 

.31 

.48 

Spatial 

.47 

.41 

.14 

-.01 

.04 

ASVAB+Spatial 

.41 

.43 

.10 

-  .06 

.  1 1 

ASVAB+Spatial+CVI  Performance 

.52 

.51 

.33 

.26 

.46 

ABLE 

.10 

.01 

.21 

.15 

.29 

ASVAB+ABLE 

.34 

.41 

.22 

.12 

.25 

ASVAB+ABLE+CVI  Performance 

.51 

.52 

.  36 

.30 

.47 

Note.  Ns  =  102-121.  Correlations  are  uncorrected  for  range  restriction. 

Coefficients  do  not  require  shrinkage  adjustments.  CVI  criterion  scores 
and  predictor  composites  were  summed. 


discriminant  validity  of  the  first-tour  job  performance  for  predicting  second- 
tour  job  performance  criteria. 

Future  analyses  of  the  LVI  Experimental  Predictor  Battery  and  LVII 
criterion  scores  will  provide  better  indications  of  the  new  predictors' 
relationships  with  second-tour  performance. 


SUMMARY  OF  PROJECT  EFFORTS  FOR  YEAR  THREE 

As  described  in  the  third  annual  report  (Campbell  &  Zook,  1994b),  the 
Project  had  four  main  objectives  during  FY  92: 
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(1)  Complete  the  Longitudinal  Second-Tour  (LVII)  data 
collection. 

(2)  Prepare  the  LVII  data  file  for  analysis. 

(3)  Analyze  the  LVII  criterion  data  to  develop  the  basic  performance 
scores . 

(4)  Model  the  covariance  structure  of  the  LVII  performance  scores. 

In  general,  because  the  LVII  results  could  be  considered  as  a  major 
replication  of  CVII,  a  great  deal  of  effort  was  devoted  to  using  the  LVII 
sample  data  in  a  confirmatory  way.  That  is,  when  possible,  the  CVII  results 
were  used  as  a  hypothesis  to  be  tested  in  LVII. 

Longitudinal  Validation  Second-Tour  Data  Collection 

The  LVII  data  collection  administered  second-tour  criterion  measures  to 
soldiers  in  the  longitudinal  validation  sample  who  had  reenlisted  for  a  second 
tour  and  who  were  available  for  assessment  at  a  specified  set  of  data 
collection  locations.  Although. this  data  collection  involved  substantially 
fewer  soldiers  than  the  CVI  or  LVI  data  collections,  it  posed  a  number  of 
special  challenges.  Having  to  locate  and  test  particular  individual  soldiers, 
especially  when  there  were  relatively  few  to  begin  with,  was  a  difficult  task. 
However,  despite  a  major  deployment  of  U.S  troops  to  Southwest  Asia  (Operation 
Desert  Shield/Storm),  LVII  data  were  collected  from  1,577  soldiers. 

A  list  of  the  instruments  administered  in  the  LVII  data  collection  is 
provided  in  Table  1.28.  Most  of  the  instruments  served  as  second-tour 
performance  criterion  measures;  several  other  instruments  (e.g.,  the 
Background  Information  Form)  provided  supplemental  data  for  the  project. 

The  original  project  plan  called  for  the  LVII  data  collection  to  take 
place  July-December  1991.  Second-tour  criterion  data  were  to  be  collected 
from  at  least  150  soldiers  in  each  of  nine  MOS  (the  Batch  A  group  designated 
in  previous  data  collections). 

Even  before  the  deployment  of  troops  to  Southwest  Asia,  the  anticipated 
data  collection  problems  included  difficulty  projecting  future  location  of 
soldiers  targeted  for  testing  because  of  frequent  reassignments,  and  limited 
access  due  to. training  or  alert  status,  leave,  and  so  forth.  The  problems 
were  compounded  by  a  tasking  system  which  requires  that  Troop  Support  Requests 
(TSRs)  be  submitted  well  in  advance  of  data  collection  (135  days  up  to  500 
days  for  U.S.  Forces  Command,  less  for  the  U.S.  Training  and  Doctrine 
Command).  Moreover,  before  detailed  data  collection  planning  activities 
began,  the  Army  was  starting  to  respond  to  directives  to  downsize  and  to 
reduce  the  proportion  of  troops  stationed  in  Germany. 

These  concerns  led  to  the  following  strategy  for  maximizing  the  number 
of  LVII  subjects.  In  May  1990,  analyses  were  conducted  to  determine  the 
number  of  Project  A  soldiers  who  were  still  in  the  Army  and  their  locations. 
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Table  1.28 


LVII  Data  Collection  Instruments _ _ _ _ _ 

Performance  Criterion  Instruments 

•  Job  Knowledge  Tests 

•  Hands-On  Tests 

•  Performance  Rating  Scales  (completed  by  supervisors) 

Army-Wide  Booklet 

MOS-Specific  Booklet 

Combat  Performance  Prediction  Scales 

Combat  Performance  Questionnaire  (Operation  Desert 

Shield/Storm),  administered  if  applicable 

•  Personnel  File  Form  (PFF) 

•  Situational  Judgment  Test  (SJT) 

•  Supervisory  Simulation  Exercises 

Personal  Counseling 
Disciplinary  Counseling 
Training 

Supplemental  Instruments 

•  Background  Information  Form 

•  MOS-Specific  Job  History  Questionnaire 

•  Supervisory  Experience  Questionnaire 

•  Army  Job  Satisfaction  Questionnaire  (AJSQ) 

•  Assessment  of  Background  and  Life  Experiences  (ABLE) 

•  Leader  and  Unit  Attitudes  Questionnaire 


It  was  clear  that  sample  size  requirements  would  not  be  met  if  only  soldiers 
having  predictor  and  first-tour  criterion  data  were  tested.  Accordingly,  the 
decision  was  made  to  test  soldiers  for  whom  predictor  and/or  first-tour 
criterion  data  were  available.  Only  soldiers  with  Project  A  data  were 
eligible.  An  additional  consideration  was  the  fact  that,  shortly  after  the 
LVI  data  collection  ended,  MOS  31C  began  declining  in  strength  because  certain 
radio  equipment  was  being  phased  out.  The  collection  of  hands-on  data  is 
inordinately  resource-intensive  for  small  numbers  of  examinees;  consequently, 
hands-on  tests  were  dropped  from  the  performance  measures  for  the  SIC 
soldiers. 

The  May  1990  analyses  also  indicated  that  appreciable  concentrations  of 
Project  A  soldiers  were  stationed  in  locations  other  than  those  identified  irf 
the  original  research  plan.  Accordingly,  requests  for  troop  support  were 
written  to  include  some  of  these  new  sites.  Then,  lists  of  all  soldiers 
eligible  for  testing  were  electronically  matched  with  each  installation's  own 
personnel  files,  to  obtain  the  most  accurate  identification  of  soldiers 
qualified  for  testing  at  each  location. 
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Data  Collection  Schedule 


The  original  research  plan,  calling  for  LVII  data  collection  July- 
December  1991,  was  adjusted  to  accommodate  the  interests  of  supporting 
commands.  It  was  agreed  that  test  sites  could  be  scheduled  to  conduct  testing 
as  early  as  May  1991  and  as  late  as  February  1992. 

This  data  collection  strategy  had  been  established  before  hostilities 
involving  U.S.  troops  in  Southwest  Asia  arose.  The  U.S.  Forces  Command,  which 
was  tasked  to  provide  the  majority  of  LVII  soldiers,  invoked  a  moratorium  on 
research  support  in  September  1990.  The  moratorium  was  lifted  in  April  1991, 
and  the  data  collection  schedule  was  again  modified.  The  final  schedule  is 
shown  in  Table  1.29.  Four  data  collection  teams  were  sent  to  Germany,  whereas 
one  team  of  data  collectors  was  sent  to  each  of  the  other  test  sites.  The 
first  LVII  data  collection  occurred  in  June  1991  and  the  last  in  July  1992. 

Table  1.29 


LVII  Data  Collection  Schedule 


Command 

Location 

Test  Dates 

1991 

USAREUR 

Germany 

7  June  -  27  June 

USAREUR 

Germany 

5  July  -  2  August 

USAREUR 

Germany 

5  July  -  3  August 

Eighth  Army 

Republic  of  South  Korea 

5  July  -  9  August 

USAREUR 

Germany 

September  -  October 

HSC 

Fort  Sam  Houston,  TX 

October 

FORSCOM 

Fort  Lewis,  WA 

9  December  -  19  December 

1992 

FORSCOM 

Fort  Drum,  NY 

13  January  -  24  January 

TRADOC 

Fort  Bliss,  TX 

20  January  -  31  January 

MDW  &  AMC 

Fort  Belvoir,  VA 

February 

TRADOC 

Fort  Knox,  KY 

2  March  -  6  March 

FORSCOM 

Fort  Bragg,  GA 

16  March  -  3  April 

TRADOC 

Fort  Benning,  GA 

31  March  -  3  April 

FORSCOM 

Fort  Riley,  KS 

6  April  -  10  April 

FORSCOM 

Fort  Hood,  TX 

4  May  -  15  May 

FORSCOM 

Fort  Campbell ,  KY 

11  May  -  15  May 

FORSCOM 

Fort  Carson,  CO 

1  June  -  5  June 

FORSCOM 

Fort  Stewart,  GA 

15  June  -  23  June 

TRADOC 

Fort  Polk,  LA 

13  July  -  16  July 

USAREUR 

U.S.  Army  Europe 

HSC 

Health  Services  Conomand 

FORSCOM 

Forces  Conmand 

TRADOC 

Training  and  Doctrine  Corrmand 

MDW 

Military  District  of  Washington 

AMC 

Army  Materiel  Command 
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Composition  of  the  teams,  in  terms  of  project  staff,  varied  from 
location  to  location.  Generally  however,  each  test  site  was  staffed  with  a 
team  comprised  of  the  following  personnel: 

1  Test  Site  Manager  (TSM) 

1-2  Hands-on  Managers  (HOMs) 

3  Test  Administrators  (TAs) 

All  of  these  positions  were  filled  by  permanent  employees  of  the  contractor 
consortium;  often  a  representative  of  the  Army  Research  Institute  was  present 
during  the  testing.  The  Army  installations  also  provided  personnel  to  help 
support  the  data  collection  activities.  In  addition  to  the  test  site  POC, 
each  test  site  provided  eight  senior  NCOs  for  each  MOS  (except  31C)  to 
administer  and  score  the  hands-on  tests,  and  two  to  four  NCOs  to  fill  general 
supporting  roles  (e.g.,  to  track  down  soldiers  who  failed  to  report  for 
testing  and  to  handle  problems  with  defective  equipment). 

Data  Collection  Team  Training 

One  day  of  classroom  training  and  considerable  follow-up  on-the-job 
training  were  provided  to  TAs  for  the  written  test  and  supervisor  rating 
procedures.  One  to  two  days  of  additional  training  were  provided  to  each  TA 
for  each  subordinate  role  a  TA  was  responsible  for  playing  in  the  Situational 
Judgment  Test.  Two  documents  were  developed  to  support  TA  training:  the  Test 
Administrator's  Manual  and  the  Supervisory  Role-Play  Exercises  Administration 
Manual. 

NCO  hands-on  scorers  were  trained  the  day  before  the  administration  of 
the  hands-on  tests  to  soldiers  in  a  given  MOS.  The  training  followed  the  same 
basic  procedures  as  those  that  had  been  used  in  the  CV  and  LVI/CVII  data 
collections  (R.  Campbell,  1985). 

Various  procedures  and  documents  were  used  to  handle  completed  data 
collection  instruments  before  they  were  shipped  to  the  facility  where  they 
would  be  processed  and  keypunched.  Test  site  personnel  checked  measures  for 
completeness  and  legibility,  and  documented  explanations  for  data  that  were 
incomplete  or  otherwise  anomalous.  Transmittal  documents  were  used  to  help 
ensure  that  data  could  be  tracked  once  it  left  the  test  site. 

After  data  collection  at  a  given  location  was  completed,  the  TSM 
prepared  and  submitted  a  report  to  ARI. 

Development  of  Basic  Scores  for  LVII  Performance  Measures 

The  LVII  performance  criterion  measures  have  been  described  in  detail 
elsewhere  (Campbell,  1989;  Campbell  &  Zook,  1990).  They  were  originally 
administered  to  second-tour  soldiers  in  the  CVII  sample  and  were  subsequently 
revised  in  preparation  for  administration  to  the  LVII  sample. 

Analyses  of  the  data  from  the  LVII  sample  had  three  major  objectives: 

(a)  examine  and  evaluate  the  psychometric  properties  of  the  LVII  measures,  (b) 
compare  the  psychometric  properties  of  the  LVII  scores  with  the  CVII  scores, 
and  (c)  develop  the  basic  criterion  scores  to  be  used  in  modeling  second-tour 
performance. 
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Job  Knowledge  and  Hands-On  Tests 


In  earlier  research  a  set  of  28-30  tasks  had  been  selected  for 
performance  measurement  in  each  MOS.  All  tasks  were  assessed  using  a  written 
job  knowledge  test  format.  Performance  on  a  subset  (14-17)  of  the  tasks  was 
assessed  using  a  hands-on  performance  test  format. 

The  full  set  of  tasks  included  (a)  common  tasks,  basic  soldiering  tasks 
that  all  soldiers  are  expected  to  know  how  to  perform  (e.g.,  first  aid, 
personal  weapons,  map  reading),  and  (b)  MOS-specific  tasks,  central  to  the 
jobs  of  the  soldiers  working  in  a  given  MOS  and  typically  unique  to  that  MOS. 

Some  tasks  are  performed  differently  depending  upon  the  type  of 
equipment  a  soldier  uses  (e.g.,  an  M16A1  rifle  versus  an  M16A2  rifle).  To 
deal  appropriately  with  such  situations,  tracked  (i.e.,  parallel)  tests  were 
prepared  for  tasks  where  the  equipment  varied. 

Before  the  CVII  measures  could  be  used  again,  technical  currency  reviews 
were  conducted.  Revisions  were  made  to  test  items  and  to  supporting  graphics 
and  handouts  as  necessary. 

Scorina  Adjustments.  Specifications  for  the  basic  scores  for  the  LVII 
job  knowledge  and  hands-on  measures  depended  largely  on  previous  work  in  CVI, 
CVII,  and  LVI.  Job  knowledge  and  hands-on  task  scores  were  calculated  as 
percent-correct  (or  percent-GO)  scores  at  all  score  levels.  The  data  for 
tracked  tests  were  examined  for  evidence  of  level  and  dispersion  differences 
between  tracks  and  no  anomalous  differences  were  found. 

■  As  with  the  previous  data  collections,  hands-on  test  scores  were 
standardized  by  site  at  the  task  level  to  control  for  site  differences.  One 
adjustment  affected  only  the  job  knowledge  tests;  that  is,  between  one  and 
four  items  per  MOS  were  dropped  because  of  doctrinal  changes  subsequent  to 
CVII. 


Table  1.30  shows  the  overall  number  of  items  in  the  job  knowledge 
component  for  each  MOS  and  the  range  of  items  per  task  test.  Table  1.31  shows 
the  overall  number  of  steps  in  the  hands-on  component  for  each  MOS  and  the 
range  of  steps  per  task  test. 

After  data  editing,  four  levels  of  scores  (Tasks,  Functional  Categories, 
Task  Factors,  and  Task  Constructs)  were  constructed.  They  are  as  depicted  in 
Figure  1.10  in  an  earlier  section  of  this  chapter. 

At  each  level  of  aggregation,  hierarchical  scores  were  computed  using 
task- level  data.  That  is,  each  category,  factor,  and  construct  score  was 
computed  by  calculating  the  mean  percentage  of  items  correct  (or  percentage  of 
steps  passed)  across  all  constituent  tasks. 

Final  Basic  Scores  for  Job  Knowledge  and  Hands-On  Measures.  The 

descriptive  statistics  calculated  across  MOS  for  both  the  Task  Construct  and 
Task  Factor  scores  do  not  differ  much  from  the  results  for  the  CVII  soldiers 
tested  (reported  in  Campbell  &  Zook,  1990). 
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Table  1.30 


Number  of  LVII  Job  Knowledge  Tasks  and  Items  by  MOS 


MOS 

No. 

of  Items 

Tasks  Dropped 

Total 

Items 

Items 

Per 

Task 

Average 
Items 
Per  Task 

IIB 

Infantryman^ 

29 

2 

128 

2-12 

4.4 

13B 

Cannon  Crewmember^ 

30 

3 

119-120 

2-8 

4.0 

19K 

Ml  Armor  Crewman 

28 

4 

142 

3-12 

5. 1 

31C 

Single  Channel  Radio  Operator^ 

30 

1 

111-112 

3-5 

3.  / 

63B 

Light  Wheel  Vehicle  Mechanic 

27 

2 

102 

2-6 

3.8 

71L 

Administrative  Specialist^ 

30 

2 

125 

2-12 

4.2 

88M 

Motor  Transport  Operator 

30 

1 

119 

3-12 

4.0 

91A/B 

Medical  Specialist 

30 

3 

113 

2-6 

3.6 

95B 

Mi  1 itary  Police 

29 

4 

109 

2-7 

3 . 8 

^  One 
the 

or  more  task  tests  were  tracked; 
same  number  of  items. 

tracked 

tests 

do  not  necessarily  have 

Table 

Number 

1.31 

■  of  LVII  Hands-On  Tasks  and 

Steps  by  MOS 

MOS 

No.  of 
Tasks 

Total 

Steps 

Steps 
Per  Task 

Average  Steps 
Per  Task 

IIB 

Infantryman 

9 

121 

5-31 

13.4 

13B 

Cannon  Crewmember® 

12 

258-259 

7-67 

21 . 5-21 . 6 

19K 

Ml  Armor  Crewman 

10 

167 

8-37 

16.7 

63B 

Light  Wheel  Vehicle  Mechanic 

:®  8 

142 

7-44 

17.8 

71L 

Administrative  Specialist*^ 

14 

140-146 

2-44 

10.0-10.4 

88M 

Motor  Transport  Operator® 

10 

193-195 

4-44 

19.3-19.5 

91A/B  Medical  Specialist 

13 

216 

6-44 

16.6 

95B 

Mi  1 itary  Pol  ice® 

10 

223-227 

7-37 

22.3-22.7 

^  One  or  more  task  tests  were  tracked;  tracked  tests  do  not  necessarily  have 
the  same  number  of  steps. 

^  One  task  was  scored  on  a  continuous  scale;  it  is  not  included  in  calculating 
total  steps,  steps  per  task,  or  average  steps  per  task. 


Task  Factor  (otherwise  known  as  CVBITS)  scores  had  been  used  in  the 
performance  modeling  exercises  conducted  for  CVI  and  LVI;  however,  Task 
Construct  scores  (i.e.,  MOS-Specific  and  General)  were  used  for  this  purpose 
in  evil.  Although  Task  Factors  preserve  somewhat  more  information  than  the 
more  highly  aggregated  Task  Construct  scores,  they  have  the  disadvantage  of 
differing  across  MOS  as  to  the  availability  of  each  of  the  six  scores.  This 
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problem  is  compounded  by  the  considerably  smaller  sample  sizes  available  for 
the  two  second-tour  data  collections  relative  to  the  two  first-tour  data 
collections.  Moreover,  in  both  CVI  and  LVI,  the  Technical  Task  Factor  score 
invariably  loaded  on  the  Core  Technical  Proficiency  performance  construct 
while  the  other  five  Task  Factor  scores  invariably  loaded  on  the  General 
Soldiering  Proficiency  performance  construct.  Therefore,  the  two  Task 
Construct  scores  were  selected  for  use  in  the  LVI I  performance  modeling 
exercise. 

Performance  Rating  Scales 

As  reported  previously  (Campbell,  1989),  the  second-tour  performance 
rating  scales  (with  the  exception  of  the  Combat  Performance  Questionnaire) 
were  developed  by  revising  and  adapting  rating  scales  developed  for  first-tour 
soldiers.  Based  on  results  of  the  CVII  data  analyses,  additional  minor 
modifications  were  made  to  these  three  sets  of  scales:  the  Army-Wide  ratings, 
the  MOS-Specific  ratings,  and  the  Combat  Performance  Prediction  scales. 

Army-Wide  Ratings.  The  Army-Wide  rating  booklet  included  12  behavior- 
based  dimensions,  seven  task-based  leadership  dimensions,  a  rating  of  overall 
effectiveness,  and  a  rating  of  senior  NCO  potential. 

Raters  in  the  CVII  sample  had  tended  to  make  frequent  use  of  the 
highest  rating  scale  values,  suggesting  that  the  scale  behavioral  anchors  may 
have  been  too  lenient  for  more  experienced  soldiers.  In  the  LVII  sample,  the 
behavioral  anchors  for  most  rating  dimensions  were  revised  to  make  the  scale 
values  reflect  a  slightly  higher  level  of  performance. 

Hos-Specific  Ratings.  The  MOS-Specific  rating  booklets  included  from  7 
to  14  technically  oriented  behavior-based  dimensions  and  a  rating  of  overall 
MOS  effectiveness.  A  set  of  scales  suitable  for  second-tour  MOS  19K  soldiers 
were  developed  by  adapting  the  second-tour  MOS  19E  scales  that  had  been  used 
in  CVII.  In  five  of  the  nine  MOS,  one  or  two  of  the  MOS-specific  dimensions 
measured  some  aspect  of  leadership  (e.g.,  Leading  the  Team  for  MOS  IIB).  As 
with  the  Army-wide  rating  dimensions,  the  CVII.  behavioral  anchors  for  most 
MOS-specific  rating  dimensions  were  revised  to  reflect  slightly  higher  levels 
of  performance. 

Combat  Performance  Prediction  Scales.  The  Combat  Performance 
Prediction  scales  consisted  of  14  items  which  depict  examples  of  soldier 
behaviors  under  combat  conditions.  The  rater's  task  was  to  estimate  the 
likelihood  that  the  ratee  would  behave  as  described  in  the  behavioral  example. 
Ratings  were  made  on  a  7-point  scale  ranging  from  very  likely  to  very 
unlikely.  The  items  were  a  subset  of  the  40  items  that  appeared  on  the 
original  CVI  version  of  the  Combat  Performance  Prediction  scales.  Unlike  the 
LVI/CVII  data  collections,  LVII  Combat  Performance  Prediction  scale  ratings 
were  collected  for  both  male  and  female  soldiers. 

Summary  of  Ratings  Data.  Across  all  nine  MOS,  two  or  more  ratings  were 
obtained  for  75  percent  of  the  soldiers  (1,194  of  1,595)  and  at  least  one 
rating  was  obtained  for  94  percent  of  the  sample  (1,494  of  1,595).  The 
soldiers  who  received  ratings  averaged  1.82  raters  per  ratee. 

Substantive  analyses  for  the  Army-wide  and  Combat  Performance 
Prediction  scale  ratings  were  carried  out  on  the  total  sample;  MOS-specific 
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ratings  were,  of  course,  analyzed  separately  by  MOS.  The  analyses  for  the 
Army-wide  and  MOS-specific  rating  scales  focused  first  on  the  distributions  of 
the  individual  ratings  and  reliability  estimates.  This  was  followed  by 
principal  factor  analyses  with  varimax  rotation  to  determine  the  composition 
of  basic  scores. 

Overall,  the  LVII  rating  distributions  were  as  expected.  The  means 
were  generally  lower  than  in  CVII  and  the  variability  was  similar.  The 
interrater  reliability  for  the  LVII  ratings  was  almost  exactly  the  same  as 
that  found  in  the  LVI  and  CVII  research. 

Several  factor  analyses  were  conducted  on  the  LVII  sample.  Army-wide 
ratings  on  the  nine  second-tour  nonleadership  dimensions  were  intercorrelated 
and  factor  analyzed  so  that  the  LVI  and  LVII  factor  structures  could  be 
compared.  The  same  procedure  was  followed  for  all  19  of  the  Army-wide 
dimensions.  For  this  analysis,  the  factor  structure  obtained  in  the  LVII 
sample  was  compared  to  the  factor  structure  obtained  in  the  CVII  sample. 

The  similarity  of  the  rotated  factor  structures  for  LVI  vs.  LVII  for 
the  nine  nonleadership/supervision  dimensions  that  are  common  to  the  first- 
tour  and  second-tour  rating  scales  was  striking. 

The  four-factor  rotated  solutions  obtained  in  the  LVII  and  CVII  samples 
for  all  19  scales  are  shown  in  Table  1.32.  Both  include  three  factors  that 
are  quite  similar  to  the  three  LVI  factors,  plus  a  separate  leadership/ 
supervision  factor. 

Basic  Scores.  Factor  analyses  of  the  Army-wide  ratings  suggest  that 
the  four-factor  CVII  solution  is  appropriate  for  LVII.  Accordingly,  the  four 
composites  shown  in  Table  1.33  and  the  overall  effectiveness  rating  were  used 
as  basic  scores  for  the  LVII  Army-wide  rating  data. 

Because  the  factor  analysis  results  did  not  indicate  multiple  factors 
for  any  of  the  MOS-specific  rating  analyses,  a  unit-weighted  composite  of  all 
dimension  ratings  for  each  MOS  was  used  as  the  basic  score.  This  is  identical 
to  the  scoring  system  used  in  CVII,  and  yielded  comparable  reliability 
estimates . 

Results  of  the  principal  component  analysis  for  the  Combat  Performance 
Prediction  scales  on  the  combined  LVII  sample  confirmed  the  findings  that  were 
obtained  in  LVI  and  CVII.  Specifically,  two  factors  were  identified;  however, 
the  second  factor  was  simply  a  reflection  of  the  first  (i.e.,  it  was  comprised 
of  the  negatively  worded  items).  The  14  items  were  summed  to  form  a  single 
basic  score. 

The  mean  LVII  Combat  Performance  Prediction  scale  composite  score  and 
standard  deviation  were  virtually  identical  to  the  mean  and  standard  deviation 
of  the  supervisor  ratings  of  soldiers  in  the  CVII  sample. 


66 


Table  1.32 


Comparison  of  LVII  and  CVII  Army-Wide  Factor  Analysis^  Results: 
All  Dimensions  _ _ _ 


Factor  Loadings  (LVII/CVII) 


Dimension 

1 

2 

3 

4 

1. 

Technical  Knowledge/Skill 

.47/. 41 

.237.24 

.267.22 

.567.65 

2. 

Effort 

.457.39 

.347.31 

.267.27 

. 58/ . 68 

3. 

Supervising 

.637.57 

.227.21 

.247.28 

.427.53 

4. 

Following  Regs/Orders 

.327.29 

.637.63 

.297.30 

.317.36 

5. 

Integrity 

.387.32 

.587.66 

.247.22 

.347.37 

6. 

Training/Development 

.607.52 

.207.24 

.277.27 

.387.52 

7. 

Maintain  Equipment 

.367.32 

.277.33 

.327.25 

.38/. 50 

8. 

Physical  Fitness 

.177.20 

.147.18 

.537.60 

.167.19 

9. 

Self-Development 

.487.41 

.297.27 

.417.44 

.327.48 

10. 

Consideration  for  Subord 

.617.47 

.407.44 

.167.26 

.287.40 

11. 

Mi  1 itary  Bearing 

.267.30 

.327.34 

.627.63 

.127.22 

12. 

Self-Control 

.167.17 

.577.56 

.207.18 

.077.09 

13. 

Role  Model 

.517.53 

.377.40 

.567.51 

.257.31  • 

14. 

Communication 

.617.62 

.397.34, 

.227.23 

.267.35 

15. 

Personal  Counseling 

.747.72 

.247.31 

.277.26 

.117.19 

16. 

Monitoring 

.687.63 

.187.31 

.307.22 

.307.41 

17. 

Organizi ng  Mi ssions/Operations 

.667.70 

.227.26 

.277.20 

.307.36 

18. 

Personnel  Administration 

.657.63 

.287.20 

.227.24 

.177.29 

19. 

Performance  Counseling 

.727.72 

.227.20 

.237.29 

.247.32 

Percent  Common  Variance 

45.3/37.6 

25.4/20.3 

18.2/16.9 

16.9/25.3 

Note.  Sample  size  is  1,388  for  LVII 

and  823 

for  evil.  CVII  analyses 

based  on 

supervisor  ratings  only. 


^  Principal  factor  analysis,  varimax  rotation. 
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Table  1.33 


Composition  of  LVII  Army-Wide  Rating  Composites 

Percent  Common 
Variance  Accounted 
for  by  Relevant  Composite 

_ Factor _ Name 

45.3  1.  Leading/Supervising  Supervising 

Training/Development 
Consideration  for  Subord 
Communication 
Personal  Counseling 
Monitoring 

Organizing  Missions/Opers 
Personnel  Administration 
Performance  Counseling 


25.4  2.  Personal  Discipline  Following  Regs/Orders 

Integrity 

Self-Control 


16.9  3.  Technical  Skill/Effort  Technical  Knowledge/Skill 

Effort 

Maintain  Equipment 


18.2  4.  Physical  Fitness/  Military  Bearing 

Military  Bearing  Physical  Fitness 


Note.  Two  dimensions  were  not  included  in  any  composites:  Acting  as  a  Role 
Model  and  Self-Development. 


Administrative  Measures:  The  Personnel  File  Form 

The  LVII  Personnel  File  Form  was  used  to  gather  self-reports  of 
archival/administrative  information  dealing  with  personnel  actions  reflective 
of  individual  performance.  The  first-tour  versions  (CVI  and  LVI)  of  the  PFF 
requested  information  regarding  (a)  evidence  of  exemplary  performance, 
including  awards  and  memoranda/certificates  of  appreciation,  commendation,  and 
achievement;  (b)  receipt  of  disciplinary  actions  (i.e..  Articles  15  and  flag 
actions);  and  (c)  test  results,  including  Physical  Readiness  test  scores, 
individual  weapon  qualification  scores,  and  Skill  Qualification  Test  scores. 

The  original  second-tour  version  of  the  PFF  developed  for  CVII  included 
these  same  types  of  variables  and  added  others.  The  additional  items  were 
related  to  education  (military  training  and  civilian  college  courses)  and 


Dimensions 

Included 
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promotions  (o.g.,  how  often  recommended  for  dccelereted  promotion,  number  of 
promotion  board  points  received).  Another  modification  was  to  distinguish 
between  awards,  memoranda,  and  disciplinary  actions  received  while  in  grades 
E-1  through  E-3  and  those  received  while  in  grades  E-4  and  above. 

Before  being  administered  to  the  LVII  sample,  the  second-tour  PFF  was 
revised  in  several  minor  ways.  Most  of  these  revisions  were  intended  to 
increase  the  interpretabi 1 ity/accuracy  of  responses  and  to  reduce  the  amount 
of  missing  data.  For  example,  the  PFF  response  format  was  changed  so  that 
soldiers  could  indicate  if  they  had  earned  more  than  one  Army  Achievement 
Medal . 


Descriptive  Statistics  and  Intercorrelations.  Means  and  standard 
deviations  for  the  administrative  indices  of  performance  are  presented  in 
Table  1.34.  The  corresponding  descriptive  statistics  for  CVII  are  not 
comparable  for  the  Awards  and  Certificates  score  because  of  response  format 
differences  between  the  CVII  and  LVII  instruments.  Otherwise,  the  means  and 
standard  deviations  for  the  LVII  and  CVII  scores  are  very  similar. 

Subsequent  analyses  suggested  that  the  basic  scores  tentatively  derived 
for  the  PFF  satisfactorily  captured  the  useful  information  on  that  form. 
Consequently,  they  were  made  available  for  use  in  the  second-tour  performance 
modeling  exercise. 


Table  1.34 


Administrative  Indices  Descriptive 

Statistics 

for  LVII  and 

CVII 

Measure 

N 

Mean 

SD 

Range 

Awards  and  Certificates^ 

CVII 

928 

10,53 

5.63 

0-44 

LVII 

1,577 

14.69 

6.79 

0-40 

Disciplinary  Actions 

CVII 

930 

.42 

.87 

0-8 

LVII 

1,577 

.37 

.76 

0-6 

Physical  Readiness  Score 

CVII 

998 

250.11 

30.68 

121-300 

LVII 

1,522 

248.81 

31.27 

23-288 

Weapon  Qualification 

CVII 

1036 

2.52 

.67 

1-3 

LVII 

1,565 

2.58 

.67 

1-3 

Promotion  Rate 

LVII 

1,513 

100.00 

7.79 

61-128 

Promotion  Rate 

CVII 

901 

100.14 

8.09 

67-121 

(CVII  Scoring) 

LVII 

1,513 

99.98 

7.48 

57-121 

®  Differences  in  LVII  and 

CVII  resu 

Its  reflect 

differences 

in  PFF 

response 

format. 
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Situational  Judgment  Test  (SJT) 

The  SJT  was  designed  to  evaluate  the  effectiveness  of  NCO  judgments 
concerninq  what  the  NCO  should  do  in  difficult  supervisory  situations.  Thus, 
jrsJ?  cL  bfviewed  as  a  job  knowledge  test  pertaining  to  the  leadership/ 
supervision  components  of  second-tour  positions. 

As  reported  previously  (Campbell,  1989),  development  of  the  SJT  involved 
askinq  qrSrof  soldiers  sVilar  to  the  target  soldiers  (i.e  ,  at  the  level 
nf  SP4/SP5/Serqeant)  to  describe  a  large  number  of  difficult  but  realistic 
sitSt  fns  thKmy  first-line  supervisors  face  on  their  jobs.  After  a  large 
numbed  of  these  situations  had  been  generated,  a  wide  variety  ot  Possible 
actions  (i  e  response  alternatives)  for  each  situation  were  gathered, 
latinos  of  "the  effLtiveness  of  each  of  these  actions  were  collected  from  both 
experts  (senior  NCOs)  and  the  target  group.  These  effectiveness  ratings  were 
used  to  select  situations  and  response  alternatives  to  be  included  on  the  SJT. 

The  initial  version  of  the  SJT,  which  was  administered  to  the  CVII 
sample  consisted  of  35  items.  Because  the  CVII  data  analysis  '"^sults 
indicated  that  the  SJT  was  a  promising  measure  of  supervisory  performance, 
Jhl^teft  waf  lengthened  to  49  items  for  the  LVII  data  collection  to  increase 
the  internal  consistency  reliability  and  make  it  easier  to  identify  SJT 
subscores. 

Because  the  SJT  had  proven  to  be  relatively  difficult 
sample  an  effort  was  made  to  select  relatively  less  dyficult  additional 
Hems  to  include  in  the  lengthened  test  for  the  LVII  administration. 

Analvsis  Procedure.  Procedures  for  scoring  the  SJT  were  identical  to 
those  used  in  CVII.  Five  different  scores  were  computed.  The  most 
Tt^^qh^forward  wa  a  simple  number  correct  score.  The  second  score  involved 
weighting  eacj;  response  alternative  by  the  mean  effectiveness  rating  given  to 
that  response  alternative  by  the  expert  group. 

Two  analogous  scoring  procedures  based  on  respondents'  choices  for  the 
least  effective  response  to  each  situation  were  also  used. 

The  fifth  score  involved  combining  the  choices  for  the  most  ^nd  the 
least  effective  response  alternative  into  one  overall  score. 
the  mean  effectiveness  of  the  response  alternative  each  soldier  chose  as 
least  effective  was  subtracted  from  the  mean  effectiveness  of  . 

alternative  they  chose  as  the  most  effective.  Because  it  is  actual lybett,|r 
to  indicate  that  less  effective  response  alternatives  are  the  least  effectiv  , 
this  score  can  be  seen  as  not  a  "difference"  score  but  a  simple  sum. 

Each  of  these  scores  was  computed  twice  for  the  LVII  soldiers, 
using  all  49  SJT  items  and  once  including  only  the  35  SJT  items  that  had 
administered  to  the  CVII  sample  as  well. 

Descriptive  statistics  and  internal  consistency  reliabilities  were 
computed  for^each  of  the  five  scoring  procedures  for  both  the 
35- item  versions  of  the  SJT.  Intercorrelations  were  computed  among  the  five 
scores  generated  by  the  five  different  scoring  procedures  for  the  49- item  SJT 
only.  Finally,  item  analyses  were  conducted  for  each  of  the  scoring 
procedures.  (See  Campbell  &  Zook,  1994b.) 
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Development  of  Subscales.  Efforts  to  identify  distinct  SJT  subscores  in 
the  evil  sample  had  not  been  particularly  successful.  However,  the  bVII 
version  of  the  SJT  contained  almost  40  percent  more  items,  and  it  was  possible 
that  a  more  interpretable  solution  would  be  found  using  the  LVII  data.  In 
addition,  a  content  analysis  of  the  SJT  items  conducted  by  Hanson  and  Borman 
(in  press)  revealed  some  promising  new  subscales.  Consequently,  the 
dimensionality  of  the  SJT  for  the  LVII  sample  was  investigated  both  rationally 
and  empirically,  with  the  primary  goal  to  develop  a  set  of  more  homogeneous 
SJT  subscores. 

The  item-level  responses  from  the  LVII  sample  were  well  distributed 
across  the  response  alternatives  for  each  item,  which  suggests  that  the 
correct  responses  to  SJT  items  were  not  obvious.  All  of  the  scoring 
procedures  resulted  in  a  reasonable  amount  of  variability  in  both  the  LVII  and 
evil  samples.  The  most  reliable  score  for  both  samples  is  M-L  Effectiveness, 
probably  because  this  score  contains  the  most  information. 

For  the  49-item  SJT,  in  which  the  maximum  possible  M-Correct  score  is 
49,  the  mean  in  the  LVII  sample  is  only  25.84,  indicating  that  this  longer 
version  of  the  SJT  was  also  relatively  difficult. 

Based  on  the  full  array  of  descriptive  statistics,  the  M-L  Effectiveness 
score  appeared  to  provide  the  best  summary  of  the  information  contained  in  the 
SJT  responses.  Thus,  all  remaining  analyses  focused  on  the  M-L  Effectiveness 
score,  which  became  the  SJT  Total  Score. 

The  rational/empirical  analysis  of  the  item  covariances  resulted  in  six 
factor-based  subscales  that  contained  between  six  and  nine  items  each.  Six 
remaining  items  were  not  included  in  any  subscale.  Definitions  of  these 
factor-based  subscales  are  presented  in  Table  1.35.  These  subscales  have 
potential  for  more  clearly  delineating  the  leadership/supervision  aspects  of 
the  second-tour  soldier  job.  They  are  included  in  one  of  the  major 
alternative  models  of  second-tour  performance  to  be  evaluated  in  subsequent 
confirmatory  analyses. 

Supervisory  Simulation  Exercises 

The  supervisory  simulation  measures  were  designed  to  assess  areas  of 
second-tour  job  performance  that  deal  with  specific  components  of  super¬ 
visor/subordinate  interaction.  These  areas  included  personal  counseling, 
disciplinary  counseling,  and  one-on-one  training.  A  trained  evaluator  (role- 
player)  acted  out  the  role  of  a  subordinate  to  be  counseled  or  trained  and  the 
examinee  assumed  the  role  of  a  first-line  supervisor  who  was  to  conduct  the 
counseling  or  training.  In  each  exercise,  evaluators  scored  the  examinees  on 
a  number  of  rating  scales. 

The  subordinate  and  supervisor  roles  were  essentially  the  same  as  those 
used  in  the  CVII  data  collection.  The  role-players  who  assumed  the  role  of 
the  subordinate  in  each  of  these  exercises  were  trained  to  play  the  roles  in  a 
standardized  fashion. 

The  rating  system  used  to  evaluate  LVII  examinees  was  modified  in 
several  ways  from  CVII.  First,  the  CVII  analyses  identified  the  scales  that 
(a)  appeared  to  be  difficult  to  rate,  (b)  conceptually  redundant,  and/or  (c) 
not  correlated  with  other  rated  behaviors  in  meaningful  ways.  These  behavior 
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Table  1.35 


Situational  Judgment  Test:  Definitions  of  Factor-Based  Subscales 


1.  Discipline  soldiers  when  necessary  (Discipline).  This  subscale  is  made 
up  of  items  on  which  the  most  effective  responses  involve  disciplining 
soldiers,  sometimes  severely,  and  the  less  effective  responses  involve 
either  less  severe  discipline  or  no  discipline  at  all.  (Six  items.) 

2.  Focus  on  the  positive  (Positive).  This  subscale  is  made  up  of  items  on 
which  the  more  effective  responses  involve  focusing  on  the  positive 
aspects  of  a  problem  situation  (e.g.,  a  soldier's  past  good  performance, 
appreciation  for  a  soldier's  extra  effort,  the  benefits  the  Army  has  to 
offer).  (Six  items.) 

3.  Search  for  underlying  reasons  (Search).  This  subscale  is  made  up  of 
items  on  which  the  more  effective  responses  involve  searching  for  the 
underlying  causes  of  soldiers'  performance  or  personal  problems  rather 
than  reacting  to  the  problems  themselves.  (Eight  items.) 

4.  Work  within  the  chain  of  command  and  with  supervisor  appropriately 
(Chain/Command).  For  a  few  items  on  this  subscale  the  less  effective 
responses  involve  promising  soldiers  rewards  that  are  beyond  a  direct 
supervisor's  control  (e.g.,  "comp"  time).  The  remaining  items  involve 
working  through  the  chain  of  command  appropriately.  (Six  items.) 

5.  Show  support/concern  for  subordinates  and  avoid  inappropriate  discipline 
(Support) .  This  subscale  is  made  up  of  items  where  the  more  effective 
response  alternatives  involve  helping  the  soldiers  with  work-related  or 
personal  problems  and  the  less  effective  responses  involve  not  providing 
needed  support  or  using  inappropriately  harsh  discipline.  (Eight  items.) 

6.  Take  immediate/direct  action  (Action).  This  subscale  is  composed  of 
items  where  the  more  effective  response  alternatives  involve  taking 
immediate  and  direct  action  to  solve  problems  and  the  less  effective 
response  alternatives  involve  not  taking  action  (e.g.,  taking  a  "wait  and 
see"  approach)  or  taking  actions  that  are  not  directly  targeted  at  the 
problem  at  hand.  (Nine  items.) 


ratings  were  dropped  and  anchors  for  some  of  the  remaining  scales  were 
changed.  The  scale  itself  was  expanded  from  3  to  5  points.  The  overall 
effectiveness  rating  was  retained,  but  the  overall  affect  and  fairness  rating 
scales  were  eliminated.  Thus,  examinees  were  rated  on  each  exercise  on  from  7 
to  11  behavioral  scales  and  on  one  overall  effectiveness  scale. 

Another  important  difference  between  the  CVII  and  LVII  measures  was  the 
background  of  the  evaluators.  The  smaller  size  of  the  LVII  data  collection 
allowed  for  the  selection  and  training  of  role-players/evaluators  who  had  been 
formally  educated  as  personnel  researchers  and  who  were  employed  full-time  by 
organizations  in  the  project  consortium.  In  contrast,  the  scope  of  the 
LVI/CVII  data  collection  had  required  the  hiring  of  a  number  of  temporary 
employees  to  serve  as  role-players. 
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DoscriptivG  analysGS  wgtg  conductGd,  followGd  by  a  SGries  of  factor 
analyses.  Maximum  likelihood  factor  analyses  with  oblique  rotations  were 
performed  within  each  exercise.  The  purpose  of  the  factor  analyses  was  to 
identify  the  content  of  basic  criterion  scores  for  each  of  the  simulation 
exercises. 

The  median  and  the  range  of  the  scale  means  and  the  median  and  the  range 
of  the  scale  standard  deviations,  for  each  exercise,  indicated  that  (a)  there 
is  a  reasonable  amount  of  variation  in  the  scale  ratings,  (b)  none  of  the 
scale  ratings  show  a  floor  effect,  and  (c)  a  reasonable  number  of  the  ratings 
do  not  show  a  ceiling  effect. 

Personal  Counseling  Exercise.  Table  1.36  presents  the  pattern  of 
matrices  resulting  from  the  factor  analyses  of  the  standardized  and  raw  score 
Personal  Counseling  exercise  ratings  that  specified  two  factors.  The  two- 
factor  structure  was  preferred  over  the  one-  or  three-  (or  more)  factor 
structures  based  on  the  superior  simple  structure  and  interpretabi 1 ity  of  the 
rotated  two-factor  pattern  matrix.  Factor  1  was  labeled  "Communication/ 
Interpersonal  Skill,"  and  Factor  2  was  labeled  "Diagnosis/Prescription." 

As  indicated  by  the  notations  in  Table  1.36,  the  factor  analysis  results 
for  LVII  did  not  exhibit  the  same  pattern  as  that  obtained  in  CVII. 

Disciplinary  Counseling  Exercise.  Table  1.37  presents  the  pattern  of 
matrices  resulting  from  the  factor  analyses  of  the  standardized  and  raw  scale 
Disciplinary  Counseling  exercise  ratings  that  specified  three  factors.  The 
three-factor  structure  was  preferred  over  the  one-,  two-,  or  four-  (or  more) 
factor  structures  based  on  the  superior  simple  structure  and  interpretabi 1 ity 
of  the  rotated  three-factor  pattern  matrix.  Factor  1  was  labeled  "Structure," 
Factor  2  was  labeled  "Interpersonal  Skill,"  and  Factor  3  was  labeled 
"Communication. " 

Training  Exercise.  Table  1.38  presents  the  pattern  matrices  resulting 
from  the  factor  analyses  of  the  standardized  and  raw  scale  Training  exercise 
ratings  that  specified  two  factors,  which  were  labeled  "Structure"  and 
"Motivation  Maintenance." 

Basic  Scores.  Scales  were  assigned  to  composite  scores  based  on  the 
patterns  of  their  relative  factor  loadings  in  the  factor  structure  for  each 
exercise.  This  procedure  resulted  in  empirically  derived  basic  scores  for 
each  exercise  that  seemed  to  have  considerable  substantive  meaning. 

Across  all  exercises,  each  basic  composite  score  was  generated  by  (a) 
standardizing  the  ratings  on  each  scale  within  each  evaluator,  (b)  scaling 
each  standardized  rating  by  its  raw  score  mean  and  standard  deviation,  and  (c) 
calculating  the  mean  of  the  transformed  (i.e.,  standardized  and  scaled) 
ratings  that  were  assigned  to  that  particular  basic  criterion  composite.  The 
ratings  were  standardized  within  evaluator  because  (a)  each  evaluator  rated 
examinees  in  only  some  MOS  and  (b)  there  was  more  variance  in  mean  ratings 
across  evaluators  than  there  was  in  mean  ratings  across  MOS.  The  standardized 
ratings  were  scaled  with  their  original  overall  means  and  standard  deviations 
so  that  each  scale  would  retain  its  relative  central  tendency  and  variability. 
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Table  1.36 


LVII 

Personal  Counseling  Exercise  Scales  and  Factor 

Analysis 

Results^ 

Scale 

Factor  1 

Factor  2 

h^ 

Communication/Interpersonal  Skill 

1. 

States  the  purpose  of  the  counseling 
session  clearly  and  concisely.^ 

.45 

-.04 

.18 

2. 

Gives  the  subordinate  positive  feedback 
for  his/her  overall  good  past 
performance. ^ 

.74 

-.10 

00 

3. 

Explains  what  the  solider  did  wrong  and 
why  it  was  or  can  be  a  problem.^ 

00 

-.06 

.12 

7. 

Maintains  eye  contact  during  the 
interview.^ 

.30 

.14 

.16 

8. 

Behaves  in  a  manner  that  demonstrate^^ 
support  and  concern  for  subordinate.** 

.52 

.30 

.54 

9. 

Conducts  the  counseling  session  in  a 
professional  manner.^ 

.47 

.12 

.29 

10. 

2 

Maintains  open  communication. 

.13 

.45  • 

.27 

Diaanosis/Prescr lotion 

4. 

Asks  open-ended,  fact  -finding  questions 
that  uncover  important  and  relevant 
information. * 

.01 

col 

.61 

5. 

Provides  advice  to  the  subordinate 
concerning  actions  that  should  be  taken  to 
solve  problems.* 

O 

I 

CO 

.73 

6. 

Sets  a  time  or  date  to  follow-up  with  the 
subordinate. * 

.01 

.52 

.27 

Omitted  Item 

11. 

Does  not  interrupt  the  subordinate  when 
he/she  is  talking.^ 

.08 

.17 

.05 

Eigenvalue*’ 

6.73 

1.39 

Note.  The  underline  indicates  which  composite  the  scale  was  assigned  to  for  the  construction  of  simulation 
exercise  basic  scores;  h^  =  Coimiunal  i  ty. 

®  Maximum  likelihood  factor  analysis  with  an  oblique  rotation.  From  analysis  of  standardized  scale  ratings. 
^  Eigenvalues  of  the  first  two  unrotated  factors. 

’  A  similar  (or  the  same)  scale  was  assigned  to  the  Personal  Counseling  -  Content  composite  score  in  CVII. 

^  A  similar  (or  the  same)  scale  was  assigned  to  the  Personal  Counseling  -  Process  composite  score  in  CVII, 

A  similar  scale  was  not  assigned  to  a  composite  score  in  the  CVII  analyses. 
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Table  1.37 


LVII  Disciplinary  Counseling  Exercise  Scales  and  Factor  Analysis  Results^ 


Scale 

Factor  1 

Factor  2 

Factor  3 

h' 

Structure 

1.  Remains  focused  on  the 

immediate  problems  (i.e.,  the 
subordinate's  absences  and/or 
lying).^ 

.38 

.12 

-.08 

.17 

2.  Determines  an  appropriate 
corrective  action. 

.57 

.04 

-.02 

.33 

3.  States  the  exact  provisions  of 
the  punishment.^ 

.57 

-.01 

.07 

.34 

Interpersonal  Skill 

6.  Conducts  the  counseling  session 
in  a  professional  manner.^ 

.07 

.72 

-.02 

.53 

7.  Defuses  rather  than  escalates 
potential  arguments.^ 

1 

O 

.67 

-.02 

.44 

Communication 

4.  Explains  the  ramifications  of 
the  soldier's  actions.^ 

.01 

-.03 

.82 

.66 

5.  Allows  the  subordinate  to 
present  his/her  view  of  the 
situation.^ 

.14 

.08 

.29 

.14 

Eigenvalue^ 

2.62 

1.52 

1.02 

Note.  The  underline  indicates  which  composite  the  scale  was  assigned  to  for  the  construction  of  simulation 
exercise  basic  scores;  h^  =  Communal i ty. 

^  Maximum  likelihood  factor  analysis  with  an  oblique  rotation.  From  analysis  of  standardized  scale  ratings. 
^  Eigenvalues  of  the  first  three  unrotated  factors.  * 

’  A  similar  (or  the  same)  scale  was  assigned  to  the  Disciplinary  Counseling  -  Content  score  in  CVII. 

^  A  similar  (or  the  same)  scale  was  assigned  to  the  Disciplinary  Counseling  -  Interpersonal  Skills  score  in 
CVII. 

A  similar  item  was  not  assigned  to  a  composite  score  in  the  CVII  analyses. 
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Table  1.38 


LVII  Training  Exercise  Scales  and  Factor  Analysis  Results^ 


Scale 

Factor  1 

Factor  2 

h' 

Structure 

2. 

Organizes  and  presents  the  training 
steps  in  a  logical  sequence. 

.64 

-.03 

.39 

3. 

Demonstrates  the  task  steps  for  the 
trainee. 

.58 

.07 

.39 

4. 

Identifies  and  corrects  the 
trainee's  errors. 

.74 

-.16 

.41 

5. 

Makes  the  trainee  practice  each 
movement  required  to  perform  the 
task . 

.66 

-.03 

.41 

6. 

Provides  specific  feedback  to  the 
trainee  following  good  performance. 

.70 

.04 

.53 

Motivation  Maintenance 

7. 

Provides  positive  feedback  to  the 
trainee  following  good  performance. 

-.01 

.81 

.65 

8. 

Encourages  the  trainee  when  mistakes 
are  made. 

-.07 

.80 

.57 

Omitted 

Items 

1. 

Presents  an  overview  of  what  will  be 
learned. 

.18 

.21 

.13 

9. 

Speaks  in  a  clear,  distinct,  and 
understandable  manner. 

.28 

.26 

.25 

Eigenvalues*’ 

6.12 

1.32 

Note.  The  underline  indicates  which  composite  the  scale  was  assigned  to  for  the  construction  of  simulation 
exercise  basic  scores.  In  the  CVII  analyses  scales  similar  (or  identical)  to  those  above  were 
assigned  to  a  single  Training  Exercise  composite  score,  h^  =  Communal ity. 

*  Maximum  likelihood  factor  analysis  with  an  oblique  rotation.  From  analysis  of  standardized  scale  ratings 
Eigenvalues  of  the  first  two  unrotated  factors. 


Summary  of  Basic  Criterion  Scores 

The  analyses  of  the  LVII  performance  measures  resulted  in  an  array  of 
basic  criterion  scores  which  were  available  for  the  performance  modeling 
analyses.  These  scores  are  summarized  in  Figure  1.12. 
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Hainds-Qn  Performance  Test 


1.  MOS-specific  task  performance  score 

2.  General  (common)  task  performance  score 

Job  Knowledge  Test 

3.  MOS-specific  task  knowledge  score 

4.  General  (common)  task  knowledge  score 

Army-Wide  Rating  Scales 

5.  Overall  effectiveness  rating 

6.  Leadership/supervision  composite 

7.  Technical  skill  and  effort  composite 

8.  Personal  discipline  composite 

9.  Physical  fitness/military  bearing  composite 

MOS-Specific  Rating  Scales 

10.  Overall  MOS  composite 

Combat  Performance  Prediction  Scales 

11.  Overall  Combat  Prediction  scale  composite 
Personnel  File  Form 

12.  Awards  and  certificates 

13.  Disciplinary  actions  (Articles  15  and  Flag  actions) 

14.  Physical  readiness 

15.  Promotion  rate 

Situational  Judgment  Test 

16.  Total  composite  or,  alternatively, 

17.  Discipline  soldiers  when  necessary 

18.  Focus  on  the  positive 

19.  Search  for  underlying  causes 

20.  Work  within  chain  of  command 

21.  Show  support/concern  for  subordinates 

22.  Take  immediate/direct  action 

Supervisory  Simulation  Exercises 

23.  Personal  counseling  -  Communication/Interpersonal  skill 

24.  Personal  counseling  -  Diagnosis/Prescription 

25.  Disciplinary  counseling  -  Structure 

26.  Disciplinary  counseling  -  Interpersonal  skill 

27.  Disciplinary  counseling  -  Communication 

28.  Training  -  Structure 

29.  Training  -  Motivation  maintenance 


Figure  1.12.  Summary  List  of  LVII  Basic  Criterion  Scores 
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The  LVII  Data  File:  Extent  of  Hissing  Data 

The  LVII  data  were  collected  from  1,595  soldiers  in  the  nine  MOS 
designated  as  Batch  A  MOS  in  previous  data  collections.  It  was  not  always 
possible  to  collect  complete  information  from  each  soldier  for  all 
instruments.  For  example,  for  the  hands-on  measures,  some  necessary  pieces  of 
equipment  might  have  been  unavailable  for  use,  making  it  impossible  to  score 
some  or  all  of  the  steps  of  a  particular  task  test,  or  supervisors  may  have 
felt  that  they  were  not  able  to  use  a  particular  rating  scale  because  of  too 
few  opportunities  to  observe  that  aspect  of  performance.  For  the  Personnel 
File  Form,  soldiers  may  have  left  questions  unanswered  if  they  did  not  know, 
or  chose  not  to  provide,  the  requested  information. 

The  number  of  soldiers  who  are  missing  all  data  on  a  particular 
instrument  can  be  determined  from  Table  1.39.  For  example,  only  341  of  the 
347  MOS  IIB  soldiers  participated  in  hands-on  testing  while  all  347  soldiers 
in  the  IIB  sample  participated  in  the  job  knowledge  test  administration. 

Various  methods  were  used  for  the  criterion  instruments  to  deal  with 
partially  missing  data.  For  the  Personnel  File  Form  and  Simulation  Exercises, 
missing  data  were  simply  left  as  missing.  For  the  other  measures,  various 
strategies  were  used  to  treat  missing  data. 

The  percentages  of  assigned  values  for  missing  data  for  each  performance 
instrument  are  shown  in  Table  1.40.  That  is,  these  are  the  individuals  in  the 
sample  who  had  some  missing  data  but  not  enough  to  be  dropped  from  the  data 
set  for  a  particular  instrument.  Instead,  their  scores  were  computed  using 
the  rules  described  in  Campbell  and  Zook  (1994b).  Note  that  these  percentages 
3ir6  gsriGTslIy  vsry  lowj  dlmost  dll  stg  Igss  thdn  oriG  pGrcGnt  GXCGpt  for  thG 
MOS  Ratings  Scales. 


Development  of  the  LVII  Performance  Model 

The  specific  objective  was  to  determine  which  model  (i.e.,  a  particular 
specification  of  the  number  of  components  and  their  substantive  content),  from 
among  several  proposed  alternative  models  of  the  latent  structure  of  basic 
criterion  score  intercorrelations,  best  fits  the  observed  data.  Analyses  were 
guided  by  the  same  general  framework  that  was  used  in  modeling  the  covariation 
among  performance  measures  for  first-tour  performance  (J.P.  Campbell  et  al., 
1990). 


One  alternative  was  the  model  developed  based  on  data  from  the  Project  A 
Concurrent  Validation  second-tour  (CVII)  sample.  This  model,  referred  to  as 
the  Training  and  Counseling  model,  is  described  in  detail  in  Campbell  and 
Oppler  (1990). 

Sample  and  Procedure 

The  sample  used  in  the  LVII  modeling  analyses  included  soldiers  from 
eight  of  the  nine  Batch  A  MOS  for  which  a  full  set  of  criterion  measures  had 
been  developed  (C.H.  Campbell  et  al.,  1990).  Because  complete  data  on  the 
entire  array  of  basic  criterion  scores  were  required  and  because  soldiers  from 
MOS  31C  did  not  have  hands-on  performance  scores,  these  soldiers  were  excluded 
from  all  of  the  present  analyses. 
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Table  1.39 


Number  of  LVII  Soldiers  With  Complete  or  Partial  Data  by  Criterion  Instrument 
and  MOS  _ _ _ _ _ 


MOS 

N 

Job 

Knowledge 

Hands- 

On 

Army-Wide 

Rating 

Scales 

MOS 

Rating 

Scales 

Combat 

Prediction 

PFF 

SJT 

Simulation 

Exercises 

IIB 

347 

347 

341 

333 

321 

313 

347 

346 

341 

13B 

180 

179 

174 

167 

164 

160 

179 

178 

174 

i9K 

168 

168 

160 

156 

149 

152 

161 

166 

156 

31C^ 

70 

70 

— 

65 

66 

64 

68 

70 

-- 

63B 

194 

192 

187 

182 

191 

176 

193 

193 

188 

71L 

157 

155 

156 

153 

150 

150 

155 

157 

156 

88M 

89 

89 

88 

86 

87 

85 

88 

89 

88 

91A/B 

222 

220 

215 

212 

208 

205 

218 

220 

214 

95B 

168 

168 

168 

167 

162 

163 

168 

168 

167 

Total 

1,595 

1.589 

1.489 

1.521 

1.498 

1.468 

1.577 

1.587 

1.485 

Note.  PFF 

=  Personnel  File  Form;  SJT 

=  Situational  Judgment  Test. 

®  Hands-On  and  Supervisory  Simulation 

Exercises  data 

were  not  collected  for  MOS  31C. 

Table  1 

Percent 

.40 

of 

LVII 

Assigned  Values 

by  Type  of 

Instrument  and 

MOS 

MOS 

Job 

Knowledge 

Hands- 

On 

Army-Wide 

Rating 

Scales 

MOS 

Rating 

Scales 

Combat 

Ratings 

Personnel 

File 

Form 

Situational 

Judgment 

Test 

Supervisory 

Simulation 

Exercises 

IIB 

.00 

.88 

.19 

2-88 

,00 

.00 

.14 

.00 

13B 

.00 

1.55 

.55 

2.00 

.00 

.00 

.17 

.00 

19K 

.00 

,00 

.03 

.68 

.00 

.00 

.06 

,00 

31C 

.00 

a 

.46 

1.79 

.00 

.00 

,15 

a 

63B 

.00 

1.92 

.54 

.66 

.00 

.00 

.12 

.00 

71L 

.00 

.92 

.85 

1.75 

.00 

.00 

.11 

.00 

88M 

.00 

.91 

.44 

3.96 

.00 

.00 

.23 

.00 

91A/B 

.00 

.92 

.78 

8,08 

.00 

.00 

.09 

.00 

95B 

.00 

.85 

.61 

6.67 

.00 

.00 

.09 

.00 

Total 
Samp  1  e 

,00 

.98 

.47 

3.33 

.00 

.00 

.12 

.00 

^  Hands-On  and  Supervisory  Simulation  Exercises  data  were  not  collected  for  MOS  31C. 
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As  a  result  of  these  considerations,  a  total  sample  of  1,144  soldiers 
with  complete  data  was  available  for  the  initial  modeling  analyses.  The  MOS 
breakdown  is  shown  in  Table  1.41. 


Table  1.41 


Number  of  LVII  Soldiers  With  Complete  Array  of  Basic  Criterion  Scores 

_  ^  ^ _ T _ \i _  kjtr\c 


(Lxc  luamg 

MOS 

LOniDdL  ren uniidriLt;  rreuiL-tiuii  uy 

Number  With 

Complete  Data 

IIB 

Infantryman 

281" 

13B 

Cannon  Crewmember 

117 

19K 

Ml  Armor  Crewman 

105 

31C 

Single  Channel  Radio  Operator 

0 

63B 

Light-Wheel  Vehicle  Mechanic 

157 

71L 

Administrative  Specialist 

129 

88M 

Motor  Transport  Operator 

69 

91A/B 

Medical  Specialist 

156 

95B 

Mi  1 itary  Police 

130 

Total 

Sample 

•  1,144 

^  These  soldiers  do  not  have  general  soldiering  scores  for  the  hands-on  or  job 
knowledge  tests. 


As  a  first  step,  several  alternative  models  of  second-tour  soldier 
performance  were  hypothesized.  The  fit  of  these  alternative  models  was  then 
assessed  using  the  LVII  data  and  compared  with  the  fit  of  the  CVII  Training 
and  Counseling  model.  Second,  because  the  Combat  Performance  Prediction 
Scales  were  not  included  in  this  initial  modeling,  key  analyses  were  rerun 
with  these  scales  included  to  confirm  that  the  Combat  scales  fit  the  models  as 
expected  and  to  determine  whether  including  them  would  affect  the  degree  of 
fit.  Once  a  best  fitting  model  was  identified,  subsequent  analyses  were 
conducted  to  determine  whether  the  model  fit  equally  well  across  MOS  and 
across  demographic  subgroups.  Finally,  based  on  the  results  of  these 
analyses,  a  set  of  criterion  construct  scores  to  be  used  in  the  LVII 
validation  analyses  was  specified. 

To  generate  alternative  hypotheses  for  the  latent  structure,  definitions 
of  the  LVII  basic  criterion  scores  used  in  the  modeling  exercise  were 
circulated  to  the  project  staff,  and  a  variety  of  hypotheses  concerning  the 
nature  of  the  underlying  structure  of  second-tour  soldier  performance  were 
obtained.  These  hypotheses  were  consolidated  into  one  principal  central 
alternative  model,  several  variations  on  this  model,  and  a  series  of  more 
parsimonious  models  that  involved  collapsing  two  or  more  of  the  substantive 
factors. 
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The  central  alternative,  the  Consideration/Initiating  Structure  model 
presented  in  Table  1.42,  differs  from  the  CVII  Training  and  Counseling  model 
primarily  in  that  it  includes  two  leadership  factors.  Based  on  staff 
judgment,  the  leadership  rating  scales  and  each  of  the  SJT  and  Supervisory 
Simulation  scores  were  assigned  to  one  of  these  two  factors. 


Table  1.42 

Cons i deration/initiating  Structure  Model 


Latent  Variable _ 

Core  Technical  Proficiency  (CT) 

General  Soldiering  Proficiency  (GP) 

Achievement  and  Effort  (AE) 


Persona  1  Discipline  ( PD ) 

Physical  Fitness/Mi  1 itary  Bearing  (PF) 

Leadership:  Initiating  Structure  (IS) 


Leadership:  Consideration  (LC) 


Written  Methods 


Ratings  Methods 


Disciplinary  Simulation 
Exercise  Methods 

Counsel ing  Simulation 
Exercise  Methods 


Scores  Loading  on  Latent  Variables _ 

MOS-Specific  Hands-On 
MOS“Specific  Job  Knowledge 

General  Hands-On 
General  Job  Knowledge 

Awards  and  Certificates 
Promotion  Rate 

Army-Wide  Ratings:  Technical  Skill/Effort  Composite 
Overall  Effectiveness  Rating 
MOS  Ratings:  Overall  Composite 
Combat  Prediction:  Overall  Composite 

Discipl inary  Actions  (reversed) 

Army-Wide  Ratings:  Personal  Discipline  Composite 

Physical  Readiness  Score 
Army-Wide  Ratings:  Physical  Fitness/ 

Bearing  Composite 

Army-Wide  Ratings:  Leading/Supervising  Composite 

SE  -  Disciplinary  Structure 

SE  -  Counseling  Diagnosis/Prescription 

SE  -  Training  Structure 

SJT  -  Disciplining 

SJT  -  linked i a te/Direct  Action 

SJT  -  Chain  of  Cormand 

SE  -  Disciplinary  Conmunication 

SE  -  Disciplinary  Interpersonal  Skill 

SE  -  Counseling  Coirmuni  cat  ion/ Interpersonal  Skills 

SE  -  Training  Motivation  Maintenance 

SJT  -  Support 

SJT  -  Search  for  Reasons 

SJT  -  Focus  on  the  Positive 

Technical  Knowledge 
Basic  Job  Knowledge 
All  Six  SJT  Scores 

All  Four  Army-Wide  Ratings  Composites 
Overall  Effectiveness  Rating 
MOS  Ratings:  Overall  Composite 
Combat  Prediction:  Overall  Composite 

All  Three  SE  -  Disciplinary  Counseling  Scores 


Both  SE  -  Personal  Counseling  Scores 


Training  Simulation  Exercise  Methods 


Both  SE  -  Training  Scores 


Because  the  within-MOS  sample  sizes  in  the  LVII  sample  were  relatively 
small  (ranging  from  69  to  281),  initial  tests  of  the  models  were  conducted 
using  the  entire  LVII  sample.  Criterion  scores  were  first  standardized  within 
each  MOS,  then  the  intercorrelations  among  these  standardized  basic  scores 
were  computed  across  all  MOS.  The  total  sample  matrix  was  used  as  input  for 
the  analyses. 

The  analysis  plan  was  to  first  compare  the  fit  of  the  Consideration/ 
Initiating  Structure  model  with  the  variations  of  this  model  and  with  the 
Training  and  Counseling  model  to  identify  the  alternatives  that  best  fit  the 
LVII  covariance  structure.  The  next  set  of  analyses  involved  comparing  a 
series  of  nested  models  to  determine  the  extent  to  which  the  observed 
correlations  could  be  accounted  for  by  fewer  underlying  factors.  LISREL  7  was 
used  to  estimate  the  parameters  and  evaluate  the  fit  of  each  of  the  alterna¬ 
tive  models. 

Results 

The  fit  of  the  Training  and  Counseling  model  in  the  LVII  sample  was 
remarkably  similar  to  the  fit  of  this  same  model  in  the  CVII  sample, 
especially  considering  that  the  performance  data  were  collected  several  years 
apart  using  somewhat  different  measures. 

Tests  of  the  Consideration/Initiating  Structure  model  and  the 
variations  on  this  model  resulted  in  a  very  poor  fit  to  the  data  (e.g.,  RMSR 
values  greater  than  .09)  and  the  program  encountered  a  variety  of  problems  in 
estimating  the  parameters  for  these  models. 

To  determine  whether  there  were  other  reasonable  alternative  models  of 
second-tour  soldier  performance,  the  LVII  total  sample  was  randomly  divided 
into  two  subsamples:  60  percent  for  model  development  and  40  percent  for 
cross-val idation/conf irmation. 

The  matrix  of  intercorrelations  among  the  basic  criterion  scores  for 
the  developmental  subsample  was  examined  by  project  staff  and  several 
alternative  models  were  suggested.  A  number  of  alternatives  tried  different 
arrangements  of  the  supervisory  simulation,  SJT,  and  rating  scale  basic 
scores,  while  still  preserving  two  leadership  factors.  None  of  the  these 
alternatives  resulted  in  a  good  fit.  However,  a  model  that  collapsed  the 
Consideration  and  Initiating  Structure  factors  into  a  single  Leadership 
factor,  included  a  single  Simulation  Exercise  method  factor,  and  moved  the 
promotion  rate  variable  to  the  new  Leadership  factor  did  result  in  a 
considerably  better  fit  to  the  data  in  both  the  developmental  and  holdout 
samples. 

The  "Leadership  Factor"  model  that  was  developed  based  on  these 
exploratory  analyses  is  shown  in  Table  1.43.  The  fit  of  the  new  Leadership 
Factor  model  to  the  LVII  data  is,  for  all  practical  purposes,  identical  to  the 
fit  of  the  Training  and  Counseling  model  to  these  same  data.  The  90  percent 
confidence  intervals  for  the  RMSEAs  overlap  almost  completely. 
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Table  1.43 


Leadership  Factor  Model 


Latent  Variable 

Scores  Loading  on  Latent  Variables 

Core  Technical  Proficiency  (CT) 

MOS-Specif ic  Hands-On 

MOS-Specific  Job  Knowledge 

General  Soldiering  Proficiency  (GP) 

General  Hands-On 

General  Job  Knowledge 

Achievement  and  Effort  (AE) 

Awards  and  Certificates 

Army-Wide  Ratings:  Technical  Skill/Effort  Composite 

Overall  Effectiveness  Rating 

MOS  Ratings:  Overall  Composite 

Combat  Prediction:  Overall  Composite 

Personal  Discipline  (PD) 

Discipl inary  Actions  (reversed) 

Army-Wide  Ratings:  Personal  Discipline  Composite 

Physical  Fitness/Military  Bearing  (PF) 

Physical  Readiness  Score 

Army-Wide  Ratings:  Physical  Fitness/Bearing  Composite 

Leadership  (LD) 

Promotion  Rate 

Army-Wide  Ratings:  Leading/Supervising  Composite 

SE  -  Disciplinary  Structure 

SE  -  Disciplinary  Comnunication 

SE  -  Disciplinary  Interpersonal  Skill 

SE  -  Counseling  Diagnosis/Prescription 

SE  -  Counseling  Communi cat  ion/ Interpersonal  Skills 

SE  -  Training  Structure 

SE  -  Training  Motivation  Maintenance 

SJT  -  Total  Score 

Written  Method 

Job-Specific  Knowledge 

General  Job  Knowledge 

SJT  -  Total  Score 

Ratings  Method 

Four  Army-Wide  Ratings  Composites 

Overall  Effectiveness  Rating 

MOS  Ratings:  Total  Composite 

Combat  Prediction:  Overall  Composite 

Simulation  Exercise  Method 

All  Seven  Simulation  Exercise  Scores 

Because  these  models  have  an  equally  good  fit  to  the  data  and  because 
the  Leadership  Factor  model  does  not  confound  method  variance  with  substantive 
variance,  the  Leadership  Factor  model  was  chosen  as  the  best  representation  of 
the  latent  structure  of  second-tour  performance. 

The  Leadership  Factor  model  was  tested  again  with  the  Combat  Performance 
Prediction  Scales  included.  For  one  comparison,  the  Combat  Prediction  Score 
was  constrained  to  load  only  on  the  Leadership  factor  and  the  Rating  Method 
factor.  For  the  second,  the  Combat  Prediction  score  was  constrained  to  load 
on  the  Achievement  and  Effort  and  the  Rating  Method  factors  only. 

The  second  assignment  (i.e.,  the  Combat  Prediction  Score  assigned  to  the 
Achievement  and  Effort  factor)  produced  a  much  better  fit. 
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Nested  Models.  Next,  the  Leadership  Factor  model  was  used  as  the 
starting  point  to  develop  a  series  of  more  parsimonious  nested  models,  similar 
to  those  tested  in  the  LVI  sample  by  Oppler,  Childs,  and  Peterson  (1994).  The 
first  was  identical  to  the  full  Leadership  Factor  model  except  that  the 
Achievement  and  Effort  factor  was  collapsed  with  the  Leadership  factor. 

Similarly,  the  second  nested  model  was  identical  to  the  model  just 
described  except  that,  in  addition,  the  Core  Technical  and  General  Soldiering 
Proficiency  factors  were  replaced  with  a  single  "can  do"  factor.  Third,  the 
Personal  Discipline  factor  and  the  new  Achievement/Leadership  factor  were  also 
collapsed.  The  fourth  model  involved  adding  the  variables  from  the  Physical 
Fitness  factor  to  this  Achievement/Leadership/Personal  Discipline  factor, 
resulting  in  a  single  "will  do"  factor.  The  final  model  collapsed  all  of  the 
substantive  factors  into  a  single  overall  performance  factor. 

Because  these  more  parsimonious  models  are  nested  within  each  other,  the 
significance  of  the  loss  of  fit  could  be  tested  by  comparing  the  chi-square 
values  for  the  various  models. 

In  the  first  nested  model,  which  involved  collapsing  the  Leadership 
factor  with  the  Achievement  and  Effort  factor,  the  resulting  decrement  in  fit 
was  very  small.  Similarly,  collapsing  the  two  "can  do"  factors  resulted  in  a 
very  small  reduction  in  model  fit.  Based  on  these  results,  a  model  with  only 
four  substantive  factors  (and  three  method  factors)  can  account  for  the  data 
almost  as  well  as  the  full  Leadership  Factor  model. 

Collapsing  additional  factors  beyond  this  level  resulted  in  larger 
decrements  in  model  fit. 

Retrospective  Re-Analysis  of  the  CVII  Data.  One  final  approach  to 
confirming  the  Leadership  Factor  model  was  to  assess  the  fit  of  this  new  model 
to  the  CVII  data.  These  results  were  virtually  identical  to  those  obtained  in 
the  LVI I  data. 

LVII  Criterion  Construct  Scores 

The  basic  criterion  construct  scores  for  use  in  validation  analyses  are 
based  on  the  full  Leadership  Factor  model,  with  six  substantive  factors.  The 
nested  model  with  four  factors  (with  a  single  Achievement/Leadership  factor 
and  a  single  "can  do"  factor  combining  Core  Technical  and  General  Soldiering 
Proficiency)  fits  the  data  almost  as  well  and  has  the  advantage  of  greater 
parsimony.  However,  it  is  still  plausible  that  all  six  performance  factors 
have  somewhat  different  antecedents  and  could  be  related  to  different 
predictor  constructs.  Therefore,  for  the  initial  validity  analyses  the  model 
that  incorporates  the  six  criterion  construct  scores  was  retained. 

Results  of  the  nested  analyses  were  used  to  form  more  parsimonious  sets 
of  criterion  construct  scores  as  well.  This  was  done  by  first  standardizing 
each  of  the  six  construct  scores  described  above  (based  on  the  full  Leadership 
model).  These  were  then  added  together  in  the  order  shown  in  Figure  1.13  to 
form  sets  of  five,  four,  three,  two  and  finally  one  criterion  composite 
construct  score. 
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Figure  1.13.  Final  LVII  Criterion  and  Alternate  Criterion  Constructs  based  on  more  parsimonious  models. 


Concluding  Comments 


In  general,  results  of  the  LVII  modeling  analyses  showed  that  both  the 
Training  and  Counseling  model  and  the  Leadership  Factor  model  fit  the  LVII 
data  quite  well.  Further,  retrospective  reanalysis  of  the  CVII  data  showed 
that  these  two  models  had  a  similarly  good  fit  in  the  CVII  sample. 

The  new  six-factor  Leadership  Factor  model  of  second-tour  performance  is 
also  consistent  with  the  CVI/LVI  model  of  first-tour  soldier  performance.  In 
addition  to  including  performance  factors  that  are  parallel  to  those 
identified  for  first-tour  soldiers,  the  LVII  second-tour  model  includes  a 
Leadership  factor  that  contains  all  measures  that  were  in  fact  targeted  at  the 
leadership/supervision  aspects  of  the  job.  This  is  consistent  with  the 
results  of  the  second-tour  job  analyses  which  indicated  that  second-tour 
soldiers  perform  many  of  the  same  tasks  as  the  first-tour  soldiers  in  addition 
to  their  supervisory  responsibilities.  In  sum,  the  Leadership  Factor  model 
provides  the  starting  point  for  the  LVII  validity  analyses  and  further 
enhances  our  understanding  of  second-tour  soldier  performance. 


ORGANIZATION  OF  THE  CURRENT  REPORT 

The  fourth  annual  report  for  the  Career  Force  Project  summarizes  a 
number  of  project  activities  that  have  been  in  the  planning  stages  for  some 
period  of  time  but  which  were  completed  during  FY  1993.  The  analysis  of 
attrition  using  "survival"  models  and  the  analysis  of  results  using  the  Army 
Job  Satisfaction  Questionnaire  fall  in  this  category.  In  addition,  the 
Project  was  called  upon  to  contribute  its  best  efforts  to  the  Joint  Service 
deliberations  concerning  the  forthcoming  revision  of  the  ASVAB.  Finally, 
following  the  specifications  for  the  LVII  performance  model  providing  a 
context  for  these  activities,  and  described  in  last  year's  annual  report,  the 
current  report  describes  the  basic  LVII  validation  analyses  and  the 
consistency  of  individual  soldier  performance  over  time.  The  chapter 
organization  is  as  follows: 

Chapter  2  describes  the  Career  Force  Project's  contribution  to  the 
deliberations  of  the  Joint  Service  Manpower  Accession  Policy  Working  Group  as 
it  developed  its  proposals  for  how  the  ASVAB  should  be  revised,  given  the 
available  data  on  the  Enhanced  Computer-Administered  Test  Battery  (ECAT).  The 
chapter  describes  an  extensive  analysis  of  all  possible  combinations  of 
subtests  relative  to  variations  in  time  constraints  and  to  changes  in  the 
criterion  measures. 

Chapter  3  reports  the  basic  validation  analyses  for  the  available 
predictors  against  the  LVII  performance  criterion  factors.  These  analyses 
parallel  those  done  for  CVI,  CVII,  and  LVI.  Results  are  compared  across 
cohorts  (1983/84  vs.  1986/87)  and  across  organizational  levels. 

Chapter  4  presents  a  brief  examination  of  the  consistency  of  performance 
from  the  training  context  to  the  first  tour  and  from  the  first  tour  to  the 
second  tour.  Since  well-defined  performance  factors  were  assessed  on  the  same 
people  at  each  stage,  it  is  possible  to  examine  the  divergent  and  convergent 
relationship  across  factors. 
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Chapter  5  reports  on  the  Project's  use  of  event  history,  or  survival, 
analyses  to  model  the  prediction  of  attrition  during  the  first  tour  of  duty. 

An  event  history  model  permits  an  examination  of  the  time  dimension  in  terms 
of  whether  the  determinants  of  attrition  are  different  for  different  lengths 
of  tenure. 

Chapter  6  summarizes  an  extensive  series  of  analyses  using  the  data 
gathered  with  the  Army  Job  Satisfaction  Questionnaire  (AJSQ).  The  job 
satisfaction  data  were  analyzed  as  a  criterion  to  be  predicted,  a  predictor  of 
attrition  and  reenlistment/non-reenlistment,  and  a  moderator  of  the  predictor/ 
criterion  relationship. 

In  sum,  the  fourth  annual  report  will  continue  the  description  of  the 
basic  validation  analyses  against  job  performance  that  began  with  the  first- 
tour  concurrent  validation  sample  (CVI)  and  will  end  with  the  second-tour 
longitudinal  validation  sample.  Direct  comparisons  can  now  be  made  across  all 
samples.  In  addition,  an  extensive  examination  of  two  alternative  criteria, 
attrition  and  job  satisfaction,  will  be  reported.  The  ECAT  analyses  will 
serve  as  an  introduction  to  the  optimal  classification  battery  analyses  that 
still  remain  to  be  done,  using  the  entire  Experimental  Battery  and  all  major 
performance  factors  as  criterion  measures. 
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Chapter  2 

IDENTIFICATION  OF  OPTIMAL  PREDICTOR  BATTERIES  USING  A  SUBSET  OF 
THE  PROJECT  A/CAREER  FORCE  EXPERIMENTAL  PREDICTOR  BATTERY 

Norman  G.  Peterson,  Christopher  E.  Sager,  Scott  H.  Oppler, 
Rodney  L.  Rosse,  and  Jennifer  L.  Crafts 


INTRODUCTION 

The  specific  analyses  reported  in  this  chapter  were  completed  to  assist 
a  subcommittee  of  the  Manpower  Accession  Policy  Working  Group  in  its 
deliberations  about  possible  revisions  to  the  ASVAB.  The  committee  had 
previously  identified  a  battery  of  experimental  tests,  designated  as  the  Joint 
Services  Enhanced  Computer  Administered  Test  battery  (ECAT),  that  would 
constitute  the  array  of  new  predictors  to  be  considered  as  potential  additions 
to  ASVAB.  Data  were  collected  across  Services  and  analyzed  by  several  sets  of 
analysts  (Kieckhaefer  et  al.,  1992;  Wolfe  et  al.  (in  preparation);  Abrahams  et 
al.,  1993;  Peterson,  Oppler,  Sager,  &  Rosse,  1993).  The  analyses  reported 
here  are  limited  to  the  experimental  predictors  in  the  ECAT  battery  that  were 
also  in  the  Project  A/Career  Force  Experimental  Battery.  The  Joint  Services 
are  also  looking  at  Navy  tests. 

The  practical  problem  is  to  decide  which  subset  of  the  ten  ASVAB 
subtests  plus  seven  tests  from  the  Project  A/Career  Force  predictor  battery 
should  be  included  in  an  improved  operational  test  battery.  This  task  is 
complicated  by  the  fact  that  a  number  of  indices  can  be  used  to  evaluate  the 
performance  of  a  test  battery.  The  indices  considered  in  this  chapter  include 
(a)  the  validity  of  the  battery,  (b)  the  battery's  capacity  to  maximize 
classification  efficiency,  (c)  the  differences  that  could  result  from  use  of 
the  test  battery  across  multiple  subgroups  (i.e..  Blacks,  Hispanics,  and 
Females),  (d)  the  performance  of  the  battery  relative  to  these  first  three 
parameters  across  multiple  MOS,  and  (e)  the  amount  of  time  required  l-o 
administer  the  battery.  The  following  sections  discuss  some  recent  studies  in 
which  each  of  these  indices  of  test  battery  performance  has  been  considered. 

Indices  for  Evaluating  the  Value  of  a  Test  Battery 


Validity 

A  recent  assessment  of  the  validity  of  the  ASVAB  is  reported  by  Oppler, 
McCloy  and  Peterson  in  Chapter  3  of  the  Career  Force  1991  Annual  Report 
(Campbell  &  Zook,  1994a).  Across  22  U.S.  Army  jobs,  the  mean  multiple 
correlation  for  predicting  end-of-training  school  knowledge  scores  with  scores 
on  the  ASVAB  subtests  was  .77.  Note  that  (a)  the  multiple  correlations  were 
adjusted  for  shrinkage  and  corrected  for  range  restriction  before  they  were 
averaged,  (b)  the  criteria  were  scored  on  written  tests  that  were  developed 
separately  for  each  job,  and  (c)  while  the  ASVAB  consists  of  10  subtests, 
analyses  performed  for  the  Army  routinely  combine  the  scores  on  the  Word 
Knowledge  and  Paragraph  Comprehension  subtests  to  form  a  single  Verbal  score, 
resulting  in  a  total  of  nine  subtest  scores. 

Oppler  et  al.  also  reported  the  incremental  validities  of  the  Career 
Force  Experimental  Battery  of  predictors.  As  summarized  in  the  preceding 
chapter,  the  Experimental  Battery  was  developed  to  measure  (a)  cognitive 
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abilities  not  already  thoroughly  assessed  by  the  ASVAB,  (b)  psychomotor 
abilities,  (c)  temperament  constructs  hypothesized  to  be  related  to  job 
performance,  and  (d)  interest  constructs  hypothesized  to  be  related  to  job 
performance.  The  results  indicate  that  while  the  Experimental  Battery  did 
result  in  incremental  validity  when  predicting  ratings  of  training 
performance,  it  did  not  show  incremental  validity  for  the  prediction  of  end- 
of-training  school  knowledge. 

In  this  chapter,  absolute  validity  is  used  as  the  index  of  validity. 
Absolute  validity  is  defined  here  as  the  multiple  correlation  associated  with 
the  regression  of  a  criterion  on  a  set  of  predictors. 

Classification  Efficiency:  Differential  Validity 

One  way  of  assigning  applicants  to  jobs  would  be  to  (a)  use  a  single  set 
of  subtest  weights  to  generate  a  predicted  performance  score  for  each 
applicant,  (b)  select  all  applicants  who  score  above  some  cutoff  on  this 
predicted  score,  and  (c)  randomly  distribute  the  selected  applicants  across 
jobs.  This  situation  will  be  referred  to  here  as  selection  without 
classification.  An  alternative  method  of  assigning  applicants  to  jobs  would 
be  to  (a)  generate  a  unique  predicted  performance  score  for  each  applicant  in 
each  job  and  (b)  assign  applicants  to  the  various  jobs  (or  to  the  Not  Selected 
group)  in  a  manner  that  maximizes  the  mean  predicted  performance  of  the 
applicants.  This  situation  will  be  referred  to  here  as  classification. 
Conceptually,  classification  efficiency  can  be  viewed  as  the  increase  in  the 
average  level  of  job  performance  that  would  result  from  replacing  the  first 
method  of  assignment  (selection  without  classification)  with  the  second  method 
of  assignment  (classification). 

A  necessary  condition  for  classification  efficiency  is  differential 
validity.  In  the  situation  where  a  unique  least-squares  regression  equation 
is  calculated  for  each  job,  differential  validity  is  the  degree  to  which  the 
subtest  weights  in  these  equations  vary  across  jobs.  Differential  validity 
results  in  intra-individual  variation  in  predicted  performance  scores  across 
jobs.  With  this  intra-individual  variation,  a  different  score  would  serve  to 
predict  an  individual's  performance  in  each  job.  Therefore,  there  would  be  a 
potential  advantage  to  optimally  matching  applicants  to  jobs  relative  to 
random  assignment  after  the  select/not  select  decision  is  made.  (This 
explanation  assumes  that  there  are  not  meaningful  mean  differences  in 
performance  across  jobs.  See  Johnson  &  Zeidner  (1990)  for  an  explanation  of 
the  potential  classification  efficiency  associated  with  mean  differences  in 
performance  across  jobs.) 

Using  a  sample  of  78,041  enlistees  in  82  U.S.  Air  Force  jobs,  Ree  and 
Earles  (1991)  assessed  differential  validity  in  the  prediction  of  training- 
school  grades  with  the  10  subtests  of  the  ASVAB.  Their  results  indicated 
that,  after  controlling  for  mean  differences  in  grades  across  jobs,  using 
unique  subtest  weights  for  each  job  resulted  in  a  multiple  correlation  of  .63; 
using  only  one  set  of  subtest  weights  for  all  jobs  resulted  in  a  multiple 
correlation  of  .61.  Ree  and  Earles  inferred  from  this  result  that  ".  .  .a 
common  equation  for  all  jobs  was  almost  as  predictive  as  an  equation  for  each 
job"  (p.  321).  A  single  equation  means  no  intra-individual  variation  in 
predicted  performance  across  jobs  and,  therefore,  no  potential  classification 
efficiency. 
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Our  index  of  differential  validity  will  be  referred  to  as  discriminant 
validity  and  defined  as  the  difference  between  mean  absolute  validity  and  mean 
general izabi 1 ity  validity.  Mean  absolute  validity  is  the  average  multiple 
correlation  resulting  from  the  regression  of  the  criterion  on  a  set  of 
predictors  in  each  of  multiple  jobs.  That  is,  the  predicted  performance 
scores  for  each  job  are  computed  using  regression  weights  that  are  specific  to 
that  job.  The  predicted  performance  scores  from  the  equation  using  the  least- 
squares  weights  computed  within  each  job  can  also  be  correlated  with 
performance  scores  in  each  of  the  other  jobs.  The  mean  of  these  correlations 
is  the  mean  general izabi 1 ity  validity.  If  there  are  (m)  jobs,  this  mean  is 
based  on  m(m-l)  validity  estimates. 

The  difference  between  mean  absolute  validity  and  mean  general izabi 1 ity 
validity  is  an  assessment  of  the  extent  to  which  there  is  intra-individual 
variation  in  predicted  performance  across  jobs.  Thus,  our  index  of 
differential  validity  is: 

Discriminant  Validity  (OV)  =  mean  absolute  validity  -  mean  general izabi 1 ity  validity 

We  will  use  this  index  of  potential  classification  efficiency  in  this  chapter 
because  of  its  close  intuitive  relationship  with  differential  validity.  It 
has  also  been  used  in  an  investigation  of  validity  in  a  prior  Army  Research 
Institute  project  (Wise,  Peterson,  Hoffman,  Campbell,  &  Arabian,  1991). 

Classification  Efficiency:  Broaden  Index  of  Classification  Efficiency 

Johnson  and  Zeidner  (1990)  argue  that  differential  validity  is  less  than 
optimal  as  an  indicator  of  potential  classification  efficiency.  They  advocate 
using  a  more  direct  estimate  of  the  benefit  derived  from  optimally  assigning 
applicants  to  jobs.  One  such  index  --  Brogden's  mean  predicted  performance 
(MPP)  (Brogden,  1959)  --  is  obtained  by  calculating 

R(l-r)^/2 

where  R  is  the  mean  of  the  absolute  validities  across  jobs,  r  is  the  mean  of 
the  correlations  among  the  predicted  scores  for  each  job,  and  f(A)  is  a 
multiplier  that  increases  with  the  selection  ratio  and  with  the  number  of  jobs 
for  which  classification  occurs  (see  Peterson  et  al.,  1993),  for  a  more 
detailed  discussion  of  the  equation  used  to  compute  mean  predicted  performance 
and  the  components  that  make  up  the  equation). 

The  formula  estimates  the  effect  of  actually  completing  the  classifi¬ 
cation  process  (e.g.,  assigning  each  individual  to  one  job)  and  then 
calculating  the  mean  of  the  predicted  performance  scores.  The  advantages  of 
the  equation  are  that  it  (a)  simultaneously  considers  indicators  of  absolute 
validity  and  differential  validity,  and  (b)  takes  into  account  two  important 
contextual  selection  variables  —  the  number  of  jobs  and  the  selection  ratio. 
Several  authors  (Brogden,  1959;  Johnson  &  Zeidner,  1990;  Statman,  1993)  have 
shown  significant  increases  in  potential  classification  efficiency  associated 
with  relatively  small  increases  in  absolute  validity  and  differential 
validity. 

However,  several  disadvantages  are  associated  with  using  Brogden's  MPP. 
There  are  underlying  assumptions  that  may  not  hold  in  all  contexts.  Two  that 
are  not  realistic  assumptions  for  the  Project  A/Career  Force  data  are  that 
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there  is  (a)  no  variation  in  R  across  jobs  (the  least-squares  regression 
equations  yield  the  same  absolute  validities  across  all  jobs),  and  (b)  no 
variation  in  r  across  jobs  (the  same  degree  of  similarity  exists  among  all 
pairs  of  equations). 

For  the  purposes  of  this  report,  we  will  explore  the  use  of  an  index  of 
predicted  performance  and  evaluate  its  performance  against  our  indices  of 
absolute  and  differential  validity.  However,  we  use  a  modified  version  of 
Brogden's  mean  predicted  performance  and  refer  to  it  as  the  Brogden  index  of 
classification  efficiency: 


R(l-r)^^^ 

where,  as  before,  R  is  the  mean  of  the  absolute  validities  across  jobs  and  r 
is  the  mean  of  the  correlations  among  the  predicted  scores  for  each  job.  The 
multiplier  f(A)  has  been  eliminated  from  this  modified  index  because  the 
number  of  jobs  and  the  selection  ratio  do  not  vary;  thus,  the  Brogden  index  of 
classification  efficiency  values  will  not  order  a  set  of  batteries  differently 
than  will  the  Brogden  MPP  values. 

There  are  an  important  similarity  and  an  important  difference  between 
differential  validity  and  the  Brogden  index.  The  similarity  is  that  R  and 
mean  absolute  validity  are  the  same  value.  The  difference  is  that  mean 
general izabi 1 ity  validity  is  the  mean  of  the  correlations  between  the  observed 
performance  scores  in  each  job  and  the  predicted  performance  scores  based  on 
the  equations  developed  in  all  of  the  other  jobs,  whereas  r  is  the  mean  of  the 
correlations  among  predicted  performance  scores  only  (i.e.,  predicted  scores 
based  on  the  equations  developed  in  each  job  are  correlated  with  predicted 
scores  based  on  the  equations  developed  in  each  of  the  other  jobs). 

Johnson  and  Zeidner  (1990)  suggest  that  computer  simulations  of 
assignments,  based  on  algorithms  designed  to  maximize  mean  predicted 
performance,  currently  provide  the  most  accurate  estimate  of  the  potential 
benefit  to  be  gained  from  classification.  Predicted  performance  scores  based 
on  empirical  or  synthetic  data  are  calculated  for  each  individual  in  each  job; 
then  an  algorithm  assigns  individuals  to  jobs  in  some  optimal  fashion,  and  the 
mean  predicted  performance  is  calculated  based  on  each  individual's  predicted 
performance  score  in  the  job  to  which  he  or  she  was  assigned. 

The  authors  point  out  that  (a)  this  method  of  estimating  potential 
classification  efficiency  does  not  depend  on  the  restrictive  assumptions 
underlying  Brogden's  MPP,  and  (b)  various  constraints  imposed  on  the 
assignment  algorithm  (e.g.,  a  different  quota  for  each  job)  do  not  reduce  the 
accuracy  of  the  estimate  of  potential  classification  efficiency.  While  it  is 
true  that  Ree  and  Earles  (1991)  demonstrated  that  the  ASVAB  subtests  produce  a 
relatively  small  amount  of  differential  validity,  Statman  (1993)  used  this 
simulation  method  to  demonstrate  that  using  the  ASVAB  subtests  to  assign 
individuals  to  jobs  would  result  in  an  increase  in  mean  predicted  performance 
relative  to  the  Army's  current  use  of  aptitude  area  composites.  The  primary 
disadvantage  of  this  method  is  that  performing  a  computer  simulation  for  each 
possible  battery  of  subtests  is  extremely  resource  intensive  and  virtually 
impossible  to  complete  for  the  kinds  of  analyses  undertaken  here. 
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In  summary,  we  will  use  two  indices  of  potential  classification 
efficiency  in  this  chapter,  the  discriminant  validity  and  the  Brogden  index  of 
classification  efficiency,  and  make  appropriate  comparisons  of  the  results. 

Subgroup  Differences 

The  Uniform  Guidelines  on  Employee  Selection  Procedures  (Equal 
Opportunity  Employment  Commission,  1978)  that  were  adopted  by  the  federal 
government  urge  that  (a)  organizations  show  that  predictors  currently  being 
used  to  select  and  classify  job  applicants  treat  subgroups  fairly,  and  (b)  the 
domain  of  individual  differences  used  to  predict  job  performance  be  expanded. 
The  latter  admonition  is  based  on  the  hypothesis  that  subgroup  differences 
might  be  reduced  by  the  discovery  and  use  of  predictors  that  demonstrate 
smaller  subgroup  differences.  In  response  to  these  recommendations,  the 
military  has  expended  considerable  effort  to  establish  the  fairness  of  the 
ASVAB  subtests  (e.g..  Wise  et  al.,  1992)  and  to  search  for  and  develop  new 
predictors  of  job  performance  (e.g.,  Peterson,  Hough,  et  al.,  1990). 

In  this  chapter,  we  examine  subgroup  differences  between  White  and  Black 
(W-B),  White  abd  Hispanics  (W-H),  and  Males  and  Females  (M-F).  Subgroup 
differences  will  be  operationally  defined  as  the  mean  of  the  predicted  scores 
for  individuals  in  the  referent  subgroup  (e.g.,  Whites)  minus  the  mean  of  the 
predicted  scores  for  individuals  in  the  non-referent  subgroup  (e.g..  Blacks) 
divided  by  the  standard  deviation  of  the  referent  subgroup's  predicted  scores. 

Issues  in  Optimizing  the  Usefulness  of  Test  Batteries 

In  addition  to  the  above,  a  number  of  other  issues  need  to  be  addressed 
when  forming  batteries  from  a  selection  of  subtests.  That  is,  what  method 
should  be  used  to  form  the  alternative  battery,  and  what  trade-offs  will  be 
faced  in  situations  where  batteries  perform  differentially  on  the  targeted 
indices? 

Results  for  a  Single  Index  of  Battery  Quality 

Research  on  absolute  validity  suggests  that  it  is  reasonable  to  expect 
that  the  ECAT  battery  subtests  might  provide  the  ASVAB  with  incremental 
validity  (McHenry  et  al.,  1990).  Research  on  classification  suggests  that 
subtests  which  measure  more  specific  cognitive  abilities  are  likely  to 
increase  the  differential  validity  of  the  ASVAB,  and  that  relatively  small 
increases  in  absolute  validity  and  differential  validity  can  have  substantial 
impact  on  potential  classification  efficiency  (Johnson  &  Zeidner,  1990). 
Research  on  subgroup  differences  suggests  that  adding  measures  of  more 
specific  cognitive  abilities  to  the  ASVAB  may  decrease  the  level  of 
differences  associated  with  the  battery  (Jensen,  1985;  Scholarios,  1990). 

This  research  does  not,  however,  emphasize  the  possibility  that  adding  a 
particular  subtest  to  the  ASVAB  might  reduce  one  type  of  subgroup  differences 
but  increase  another  type. 

Evaluation  of  Predictor  Batteries  Using  Multiple  Indices 

Broadening  the  domain  of  cognitive  abilities  currently  assessed  by  the 
ASVAB  could  simultaneously  increase  the  battery's  level  of  absolute  validity 
and/or  differential  validity  and  decrease  the  level  of  subgroup  differences 
(e.g.,  Scholarios,  1990;  Johnson  &  Zeidner,  1990).  The  rationale  for  this 
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prediction  is  based  on  three  observations:  (a)  Genera!  cognitive  ability 
accounts  for  a  substantial  portion  of  the  variance  in  the  ASVAB's  current 
subtests  (Ree  &  Earles,  1991),  (b)  measures  of  specific  cognitive  abilities 
tend  to  show  smaller  subgroup  differences  than  measures  of  general  cognitive 
ability  (Jensen,  1985),  and  (c)  measures  of  specific  cognitive  abilities  would 
increase  the  absolute  and  differential  validity  of  the  current  ASVAB  (McHenry 
et  al.,  1990;  Johnson  &  Zeidner,  1990). 

However,  as  noted  earlier,  a  potential  problem  not  emphasized  in  prior 
research  is  that  measures  of  specific  abilities  that  reduce  differences 
between  one  set  of  subgroups  might  increase  them  for  another  set.  For 
example,  adding  a  measure  of  a  particular  psychomotor  ability  might  reduce 
differences  between  White  and  Blacks,  but  increase  them  between  Males  and 
Females. 

Absolute  Validity  Versus  Classification  Efficiency 

How  does  one  choose  an  "optimal"  battery  if  the  combination  of  subtests 
that  maximizes  absolute  validity  is  not  the  same  as  the  combination  of 
subtests  that  maximizes  potential  classification  efficiency?  Brogden  (1959) 
indicates  that  substantial  gains  in  classification  efficiency  can  be  achieved 
with  relatively  small  increases  in  absolute  validity  and  differential 
validity;  however,  there  are  trade-offs  between  these  two  parameters  (Statman, 
1993).  If  all  the  assumptions  are  met,  Brogden's  MPP  can  determine  which 
combination  of  subtests  would  result  in  the  greatest  mean  predicted 
performance.  However,  because  Brogden's  assumptions  are  rarely  met  (and  are 
not  met  by  the  data  analyzed  in  this  report),  his  equation  for  estimating  MPP 
is  of  limited  usefulness  for  choosing  among  alternative  batteries  that 
maximize  absolute  validity  versus  maximizing  differential  validity. 

Johnson  and  Zeidner' s  method  of  computer  simulations  is  capable  of 
addressing  this  issue  (Scholarios,  1990).  Scholarios  based  a  computer 
simulation  on  the  Project  A  data  set.  The  results  of  the  simulation  indicated 
that  when  the  criterion  was  core  technical  proficiency  and  the  predictors  were 
the  ASVAB  subtests  and  the  Project  A  experimental  test  battery,  test  batteries 
that  were  chosen  based  on  maximizing  differential  validity  resulted  in  higher 
mean  predicted  performance  than  test  batteries  that  were  chosen  based  on 
maximizing  absolute  validity.  It  is  important  to  note  that  these  simulations 
involved  relatively  few  potential  combinations  of  subtests,  as  compared  to  the 
number  that  will  be  considered  in  this  chapter. 

An  Alternative  Method  for  Identifying  "Optimal"  Batteries 

A  variation  on  a  stepwise  regression  method  was  used  by  Abrahams,  Pass, 
Kusulas,  Cole,  and  Kieckhaefer  (1993)  to  identify  optimum  combinations  of  10 
ASVAB  and  9  ECAT  subtests  to  maximize  absolute  validities  of  batteries  ranging 
in  length  from  1  to  19  subtests. 

For  each  analysis,  the  steps  were  as  follows:  (a)  The  single  test  that 
had  the  highest  weighted  average  validity  across  schools  (jobs)  was  selected, 
(b)  multiple  correlations  were  computed  for  all  possible  combinations  of  the 
first  test  with  each  of  the  remaining  tests,  and  (c)  the  average  multiple 
correlation  was  then  computed  for  each  test  pair  and  the  combination  with  the 
highest  weighted  average  was  selected.  This  process  was  repeated  until  all 
remaining  tests  were  included.  The  idea  is  that  at  each  step,  the  subtests 
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currently  in  the  prediction  equation  represent  the  test  battery  of  that  length 
that  maximizes  average  absolute  validity. 

Three  limitations  are  associated  with  this  method.  First,  because  this 
method  does  not  allow  subtests  to  be  dropped  at  later  stages  of  accretion 
(i.e.,  the  procedure  uses  forward  stepwise  regression  only),  the  particular 
combination  of  subtests  at  the  n*”  step  may  not  be  the  battery  of  n  tests  that 
maximizes  mean  absolute  validity.  The  relative  contribution  of  a  particular 
subtest  may  diminish  as  other  subtests  are  added  to  the  battery. 

Second,  this  method  seeks  only  to  identify  the  optimal  battery  relative 
to  the  mean  absolute  validity  across  jobs;  it  does  not  account  for  potential 
classification  efficiency  or  differences  among  subgroups.  One  implication  of 
this  difficulty  is  that  for  a  test  battery  of  any  length,  the  combination  of 
subtests  identified  by  this  method  may  produce  a  relatively  low  level  of 
differential  validity  and  relatively  high  subgroup  differences  compared  to 
another  combination  of  subtests  with  a  similar  level  of  mean  absolute 
validity,  but  a  higher  level  of  differential  validity  and  lower  subgroup 
differences. 

A  final  limitation  of  this  method  is  that  while  the  number  of  subtests 
in  a  test  battery  is  related  to  the  cost  of  administering  the  battery,  the 
actual  test  battery  administration  time  might  provide  a  more  accurate 
assessment  of  cost. 

Suiwnarv  of  Issues 

The  goal  of  the  analyses  reported  here  is  to  identify  an  "optimal" 
combination  of  tests  selected  from  the  ASVAB  subtests  and  Project  A/Career 
Force  ECAT  predictors  for  predicting  three  criteria  (technical/school 
knowledge,  on-the-job  core  technical  proficiency,  and  on-the-job  hands-on  test 
performance)  at  various  time  lengths  for  test  battery  administration.  In  the 
context  of  this  study,  "optimal"  is  defined  as  the  maximization  of  three 
distinct  types  of  battery  performance  indices. 

However,  there  are  likely  to  be  trade-offs  involved  when  building 
batteries.  There  is  no  established  way  to  choose  between  two  batteries  when 
one  battery  has  greater  absolute  or  differential  validity  and  another  battery 
has  smaller  subgroup  differences,  or  between  two  batteries  when  one  decreases 
one  type  of  subgroup  difference  (e.g.,  W-B)  and  the  other  decreases  another 
type  (e.g.,  M-F).  Yet,  if  the  relative  standings  of  alternative  batteries  on 
the  various  test  battery  performance  indices  are  known,  then  it  may  be 
possible  to  make  explicit  judgments  about  which  battery  to  use. 


A  NEW  APPROACH 

It  is  not  possible  to  choose  a  combination  of  subtests  that 
simultaneously  optimizes  all  the  relevant  indications  of  test  battery  quality. 
It  is  possible,  however,  to  calculate  all  the  indices  of  test  battery 
performance  for  every  possible  combination  of  subtests  that  fall  within  a 
given  test  administration  time  interval.  For  example,  this  method  can  be  used 
to  compute  the  absolute  validity,  differential  validity,  Brogden's  index,  W-B, 
W-H,  and  M-F  differences  of  every  combination  of  subtests  that  require  from 
134  to  164  minutes  of  administration  time.  These  combinations  of  subtests  can 
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then  be  rank-ordered  according  to  each  index  of  test  battery  performance.  For 
instance,  the  top  20  test  batteries  ranked  on  the  basis  of  maximum  absolute 
validity' can  be  compared  to  the  top  20  test  batteries  ranked  on  the  basis  of 
minimum  M-F  differences. 

An  advantage  of  this  method  is  that  it  provides  explicit  information 
necessary  to  evaluate  trade-offs:  If  subtests  are  included  to  optimize  test 
battery  performance  on  one  index,  how  will  the  battery  perform  on  the  other 
indices? 

This  approach  was  one  of  those  followed  in  the  Joint  Services  Enhanced 
Computer  Administered  Test  validation  study  designed  to  evaluate  different 
combinations  of  19  ASVAB  and  ECAT  subtests  in  predicting  end-of-traimng 
performance  (Peterson  et  al.,  1993).  Data  were  collected  from  9,037  Air 
Force,  Army,  and  Navy  enlistees  representing  17  military  jobs.  The  analysis 
procedures  used  three  time  intervals,  included  two  ASVAB  subtests  (Arithmetic 
Reasoning  and  Word  Knowledge)  in  every  potential  test  battery  in  each  of  the 
three  time  intervals,  and  evaluated  absolute  validity,  differential  validity, 
classification  efficiency,  and  three  types  of  subgroup  differences  (W-B,  W-H, 
and  M-F)  for  each  battery  in  each  time  interval. 

Results  indicated  that  no  single  test  battery  (within  each  time 
interval)  simultaneously  optimized  all  the  test  battery  indices  examined.  The 
researchers  identified  trade-offs  associated  with  maximizing  absolute  validity 
or  classification  efficiency  vs.  minimizing  all  three  types  of  subgroup 
differences,  and  with  minimizing  M-F  differences  vs.  minimizing  either  W-B  or 
W-H  differences. 

The  same  approach  was  used  to  obtain  the  results  reported  in  this 
chapter.  However,  the  analyses  here  were  done  for  each  of  three  different 
Project  A/Career  Force  crtierion  measures  and  the  sample  was  limited  to  Army 
personnel  only. 


Objectives 

Two  primary  objectives  were  addressed  in  the  analyses  reported  in  this 
chapter.  The  first  was  to  examine  the  battery  indices  for  each  potential  test 
battery  at  three  time  intervals  that  are  much  shorter,  somewhat  shorter,  and 
approximately  the  same  length  as  the  current  ASVAB.  These  lengths  were  chosen 
pragmatically;  it  is  unlikely  that  additional  time  can  be  made  available  for 
military  testing,  but  shorter  time  periods  could  be  viewed  as  desirable.  The 
second  objective  was  to  explore  the  trade-offs  associated  with  choosing  test 
batteries  that  individually  optimize  each  of  the  six  parameters  of  test 
battery  performance. 

We  used  an  approach  similar  to  that  described  for  the  ECAT  study  above 
(Peterson  et  al.,  1993).  The  same  analysis  procedure  was  followed;  we  used 
three  distinct  battery  time  intervals  and  included  the  Arithmetic  Reasoning 
and  Word  Knowledge  ASVAB  subtests  in  every  potential  test  battery  for  each  of 
the  three  time  intervals. 
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Sample 


The  data  were  from  nine  of  the  Longitudinal  Validation  (LV)  first-tour 
Batch  A  MOS  (19E  was  excluded);  they  are  listed  in  Table  2.1.  Recall  that 
these  data  were  collected  using  a  longitudinal  validation  design  with 
approximately  two  years  between  the  time  when  predictor  tests  were 
administered  to  soldiers  in  the  sample  and  the  time  when  criterion  data  were 
collected  on  their  job  performance.  The  end-of-course  training  performance 
technical  knowledge  test  was  administered  3-6  months  after  the  individual 
started  basic  training. 

Table  2.1 


MOS  Sample  Sizes  for  Calculating  Absolute,  General izabi 1 ity,  and  Discriminant 
Validities  of  Test  Batteries 


Criterion 

MOS 

Technical/School 

Knowledge 

On-the-Job  Core 
Technical  Proficiency 

On-the-Job  Hands- 
On  Performance 

IIB 

2,210 

292 

292 

13B 

3,356 

696 

696 

19K 

1,555 

543 

543 

31C 

564 

206 

206 

63B 

936 

510 

510 

71L 

1,257 

305 

305 

88M 

1,084 

273 

273 

91A 

2,956 

622 

622 

95B 

3,401 

312 

312 

Measures 


Criteria 


Three  LV  first-tour  criteria  were  used  in  these  analyses.  The  first  was 
Technical/School  Knowledge,  an  end-of-training  criterion.  The  other  two  were 
on-the-job  criteria:  Core  Technical  Proficiency  and  Hands-On  Test 
Performance.  Sample  sizes  for  each  of  the  three  criterion  measures  are  listed 
in  Table  2.1.  A  total  of  17,319  soldiers  were  tested  for  the  Technical/School 
Knowledge  criterion,  and  3,759  were  tested  for  the  criteria  of  Core  Technical 
Proficiency  and  Hands-On  Test  Performance. 

Predictors 


Our  Career  Force  analyses  were  designed  to  identify  potential 
combinations  of  15  candidate  predictors  to  be  administered  along  with  the 
Arithmetic  Reasoning  and  Word  Knowledge  subtests  of  the  ASVAB.  These  two 
subtests  were  designated  as  the  base  condition;  as  such,  they  were  included  in 
each  of  the  potential  test  batteries  discussed  below.  The  15  predictors 
considered  included  all  the  remaining  ASVAB  subtests  and  the  Experimental 
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Battery  subtests  under  serious  consideration  for  supplementing  the  ASVAB. 
Specifically,  the  tests  were: 

•  Eight  ASVAB  subtests  other  than  Arithmetic  Reasoning  (ASAR)  and 
Word  Knowledge  (ASWK): 

Auto  and  Shop  Information  (ASAS) 

Coding  Speed  (ASCS) 

Electronics  Information  (ASEI) 

General  Science  (ASGS) 

Mechanical  Comprehension  (ASMC) 

Mathematical  Knowledge  (ASMK) 

Numerical  Operations  (ASNO) 

Paragraph  Comprehension  (ASPV) 

•  Three -Project  A/Career  Force  paper-and-penci 1  spatial  tests: 

Assembling  Objects  (SPAO) 

Orientation  (SPOR) 

Reasoning  (SPRS) 

•  Three  Project  A/Career  Force  computerized  tests: 

Target  Identification  (Decision  Time)  (CMTI) 

One-Hand  Tracking  (CMIT) 

Two-Hand  Tracking  (CM2T) 

•  One  Project  A/Career  Force  computerized  test  composite: 

Short-Term  Memory  (CMST) 

These  subtests  are  described  in  greater  detail  in  Peterson,  Russell,  et  al. 
(1990). 

Test  Battery  Time  Intervals 

Three  different  test  battery  time  intervals  were  related  for  this  study. 
These  intervals  were  defined  by  total  minutes  allowed  for  "start-up"  time, 
instruction  time,  and  test-taking  time.  The  three  time  intervals  were:  74- 
104  minutes,  134-164  minutes,  and  194-224  minutes.  Start-up  time  was 
specified  to  last  20  minutes,  regardless  of  the  number  or  types  of  tests  to  be 
included  in  the  battery,  and  instruction  time  was  set  at  3  minutes  per  test. 
The  amount  of  time  required  to  take  each  test,  not  including  instructions,  is 
listed  in  Table  2.2. 


Analyses 

The  following  analyses  were  conducted  for  each  of  the  three  criterion 
measures  and  three  time  intervals  specified  above. 

Absolute  Validity 

The  multiple  correlation  that  was  associated  with  all  possible 
combinations  of  predictors  for  which  the  total  test  administration  time  was 
within  the  specified  time  interval,  and  that  included  the  Arithmetic  Reasoning 
and  Word  Knowledge  subtests,  was  computed  within  each  MOS.  For  example,  there 
are  10,152  unique  combinations  of  predictors  for  which  the  total  test 
administration  time  is  between  134  and  164  minutes  (and  which  include  the 
Arithmetic  Reasoning  and  Word  Knowledge  subtests).  Therefore,  10,152  multiple 
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Table  2.2 


Amount  of  Time  Required  for  Each  Test 


Test 

Title 

Source 

Length 

(minutes)^ 

ASAR*’ 

Arithmetic  Reasoning 

ASVAB 

36 

ASWK^’ 

Word  Knowledge 

ASVAB 

11 

ASAS 

Auto  and  Shop  Information 

ASVAB 

11 

ASCS 

Coding  Speed 

ASVAB 

7 

ASEI 

Electronic  Information 

ASVAB 

9 

ASGS 

General  Science 

ASVAB 

11 

ASMC 

Mechanical  Comprehension 

ASVAB 

19 

ASMK 

Mathematical  Knowledge 

ASVAB 

24 

ASNO 

Numerical  Operations 

ASVAB 

3 

ASPC 

Paragraph  Comprehension 

ASVAB 

13 

SPAO 

Assembling  Objects 

Spatial 

18 

SPOR 

Orientation 

Spatial 

10 

SPRS 

Reasoning 

Spatial 

12 

CMST 

Short-Term  Memory 

Computer 

5 

CMTI 

Target  Identification 

Computer 

3 

CMIT 

One-Hand  Tracking 

Computer 

4 

CM2T 

Two-Hand  Tracking 

Computer 

4 

Does  not  include  3  minutes  on  each  test  for  instructions. 
^  Included  in  all  test  batteries. 


correlations  were  computed  within  each  MOS  (a  different  multiple  correlation 
for  each  combination  of  predictors  meeting  the  above  requirements). 

These  multiple  correlations  were  corrected  for  multivariate  range 
restriction  using  the  Lord  and  Novick  (1968)  formula  and  the  10  x  10  ASVAB 
covariance  matrix  from  the  1980  Youth  Population  (Department  of  Defense, 

1982).  They  were  also  adjusted  for  shrinkage  using  Rozeboom's  (1978) 

Formula  8.  The  unweighted  average  multiple  correlations  across  the  nine  MOS 
for  each  combination  are  referred  to  as  the  mean  absolute  validities. 

Potential  Classification  Efficiency;  Discriminant  Validity 

The  least-squares  equation  computed  within  each  MOS  (for  each  possible 
combination)  was  also  used  to  predict  performance  in  each  of  the  other  MOS. 
That  is,  for  each  unique  test  combination,  each  of  the  nine  MOS  equations  was 
applied  to  the  other  eight  MOS.  This  resulted  in  the  computation,  for  each 
unique  test  battery,  of  72  correlations  between  predictor  equations  developed 
in  one  MOS  and  performance  in  a  different  MOS.  Note  that  these  correlations 
were  not  adjusted  for  shrinkage  (because  there  is  no  capitalization  on 
chance) . 

The  unweighted  average  of  these  72  correlations  for  each  combination  is 
referred  to  as  the  mean  general izabi 1 ity  validity,  and  the  difference  between 
the  mean  absolute  validity  and  the  mean  general izabi 1 ity  validity  is  referred 
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to  as  the  mean  discriminant  validity.  This  latter  index  provides  an 
indication  of  the  extent  to  which  the  regression  weights  associated  with  a 
particular  combination  of  predictors  vary  across  MOS. 

Potential  Classification  Efficiency;  Broaden  Index 

The  Brogden  index  for  the  nine  MOS  was  calculated  for  each  combination 
of  subtests  that  included  Arithmetic  Reasoning  and  Word  Knowledge,  as  follows: 

Mean  absolute  validity  x  [1  -  mean  correlation  among  the  predicted  scores]’^® 


Subgroup  Differences 

Finally,  three  estimates  of  subgroup  differences  (i.e..  White  vs.  Black; 
White  vs.  Hispanic;  Male  vs.  Female)  were  computed  for  each  possible 
combination  of  predictors.  This  was  done  by  applying  the  least-squares 
regression  equations  computed  within  each  MOS  (discussed  above)  to  all 
soldiers  in  the  LV  predictor  sample.  (The  LV  predictor  sample  included  25,538 
whites,  9,058  blacks,  1,252  Hispanics,  32,393  males,  and  4,722  females.)  Each 
equation  generated  a  composite  score  for  each  soldier. 

Next,  mean  composite  scores  were  calculated  by  subgroup  for  each 
equation.  Estimates  of  subgroup  differences  were  computed  for  each  equation 
by  subtracting  the  non-referent  subgroup  (e.g.,  Blacks)  mean  composite  score 
from  the  referent  subgroup  (e.g..  Whites)  mean  composite  score  and  dividing 
this  value  by  the  standard  deviation  of  the  referent  subgroup's  composite 
scores.  These  estimates  were  then  averaged  across  equations,  resulting  in 
three  indices  of  subgroup  differences  (i.e.,  White  vs.  Black;  White  vs. 
Hispanic;  Male  vs.  Female).  An  advantage  of  this  approach  is  that  the 
estimates  of  subgroup  differences  were  relatively  stable  because  they  were 
calculated  on  a  relatively  large  sample. 


RESULTS 

The  results  are  summarized  in  Table  2.3.  The  separate  sections  of  the 
table  present  the  general  summary  statistics  and  information  about  the 
performance  of  the  prediction  equations  across  all  possible  combinations  of 
predictors  (that  each  include  Arithmetic  Reasoning  and  Word  Knowledge)  for 
each  time  interval  and  for  each  of  the  criterion  measures. 

The  first  three  rows  report:  (a)  the  number  of  possible  combinations  of 
tests  for  each  time  limit,  (b)  the  mean  (and  standard  deviation)  of  the 
testing  times  associated  with  the  combinations  for  each  time  limit,  and  (c) 
the  mean  (and  standard  deviation)  of  the  number  of  tests  associated  with  those 
combinations.  For  example,  the  number  of  combinations  of  tests  that  fit  into 
the  134-164  minute  interval  is  10,152.  The  average  amount  of  time  associated 
with  these  combinations  is  151.0  (SO  =  8.7)  minutes,  and  the  average  number  of 
tests  is  8.2  (SD  =  1.0). 

Average  Battery  Performance  Indices  for  Each  Criterion 

The  remainder  of  Table  2.3  contains  the  average  score  on  each  of  the  six 
predictor  battery  performance  indicators  for  each  of  the  three  criteria  and 
each  time  interval.  The  six  indicators  are  (a)  the  mean  absolute  validity 
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Table  2.3 

Summary  of  Results  for  Predicting  Each  Criterion  at  Each  Time  Interval 


Genera  1 

Sunmary  Statistics 

74-104 

Minute 

Batteries 

134-164 

Minute 

Batteries 

194-224 

Minute 

Batteries 

Total  Number  of  Combinations 

210 

10,152 

6.509 

Mean  Testing  Time  in  Minutes  (SD) 

97.5 

(5.8) 

151.0 

(8.7) 

205,7 

(8.4) 

Mean  Number  of  Tests  (SO) 

4.6 

(.7) 

8.2 

(1.0) 

11.6 

(1.0) 

Technical/School  Knowledge 

Mean  Validity  Coefficient  (SD) 

.739 

( .009) 

.761 

(.007) 

.771 

(.005) 

Mean  Discriminant  Validity  Index  (SD) 

.013 

(.008) 

.025 

(.007) 

.031 

(.004) 

Mean  Brogden  Index  (SD) 

.102 

(.026) 

.145 

(.018) 

.164 

(.010) 

Mean  W-B  Difference  (SD) 

1.426 

(.054) 

1.457 

(.045) 

1.465 

(.028) 

Mean  W-H  Difference  (SD) 

1.053 

(.042) 

1.080 

(.034) 

1.088 

(.023) 

Mean  M-F  Difference  (SD) 

.262 

(.089) 

.315 

(.069) 

.321 

(.044) 

On-the-Job  Core  Technical  Proficiency 

Mean  Validity  Coefficient  (SO) 

.593 

(.010) 

.615 

(.009) 

.623 

(.006) 

Mean  Discriminant  Validity  Index  (SD) 

,012 

(.007) 

.022 

(.007) 

,027 

(.005) 

Mean  Brogden  Index  (SD) 

.123 

(.018) 

.163 

(.015) 

.186 

(.010) 

Mean  W-B  Difference  (SD) 

1.339 

(.063) 

1.378 

(.047) 

1.392 

(.029) 

Mean  W-H  Difference  (SD) 

.993 

(.051) 

1.019 

(.043) 

1.023 

(.030) 

Mean  M-F  Difference  (SD) 

.342 

(.114) 

.407 

(.092) 

.411 

(.057) 

On-the-Job  Hands-On  Test  Perfomance 

Mean  Validity  Coefficient  (SD) 

.455 

(.017) 

.490 

(.012) 

.500 

(.007) 

Mean  Discriminant  Validity  Index  (SD) 

,017 

(.007) 

.027 

(.007) 

.030 

(.006) 

Mean  Brogden  Index  (SD) 

.132 

(.014) 

.167 

(.012) 

.187 

(.010) 

Mean  W-B  Difference  (SD) 

1.182 

(.103) 

1.197 

(.065) 

1.193 

(.035) 

Mean  W-H  Difference  (SD) 

.888 

(.083) 

.899 

(.055) 

.895 

(.034) 

Mean  M-F  Difference  (SD) 

.526 

(.175) 

.636 

(.125) 

.656 

(.074) 

W-B  =  White-Black:  W-H  =  White-Hispanic;  M-F  =  Male-Female. 
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coefficient,  (b)  the  mean  discriminant  validity  index,  (c)  the  mean  Brogden's 
index  of  classification  efficiency,  (d)  the  mean  White  vs.  Black  difference, 
(e)  the  mean  White  vs.  Hispanic  difference,  and  (f)  the  mean  Male  vs.  Female 
difference. 

For  example,  the  average  validity  for  predicting  Technical/School 
Knowledge  was  .739  (SD  =  .009)  across  the  210  test  combinations  fitting  into 
the  74-104  minute  time  interval.  The  average  discriminant  validity  was  .013 
(SD  =  .008);  the  average  Brogden's  index  was  .102  (SD  =  .026);  the  average 
White  vs.  Black  difference  was  1.426  (SD  =  .054);  the  average  White  vs. 
Hispanic  difference  was  1.053  (SO  =  .061);  and  finally  the  average  Male  vs. 
Female  difference  was  .262  (SD  =  .089). 


Correlations  Among  Performance  Indicators  for  Each  Criterion 
for  Potential  Batteries  at  Each  Time  Interval 

Tables  2.4  through  2.12  contain  the  correlations  among  the  mean 
predictor  battery  performance  indicator  scores  for  each  criterion  and  each 
time  interval.  For  example,  Table  2.8  shows  that  when  predicting  On-the-job 
Core  Technical  Proficiency  at  the  134-164  minute  time  interval  (across  the 
10,152  combinations  of  predictors),  the  average  correlation  (across  the  nine 
MOS)  between  discriminant  validity  and  White  vs.  Black  differences  is  .36. 
This  means  that,  for  this  criterion/time  interval,  combinations  of  predictors 
with  relatively  low  discriminant  validity  tended  to  have  relatively  low  White 
vs.  Black  differences,  and  combinations  of  predictors  with  relatively  high 
discriminant  validity  tended  to  have  relatively  high  White  vs.  Black 
differences. 


Table  2.4 

Correlations  Among  Mean  Predictor  Battery  Performance  Indicator  Scores  Across 
210  Combinations  for  Technical/School  Knowledge:  74-104  Minute  Interval _ 


Indicators 

Val. 

Ois.  V 

BI 

W-B 

W-H 

M-F 

Validity 

1.00 

Discriminant  Validity 

0.88 

1.00 

Brogden  Index 

0.87 

0.99 

1.00 

W-B  Difference 

0.48 

0.51 

0.45 

1.00 

W-H  Difference 

0.41 

0.35 

0.31 

0.86 

1.00 

M-F  Difference 

0.78 

0.87 

0.88 

0.35 

0.24 

1.00 

W-B  =  White-Black;  W-H 

=  White- 

•Hispanic; 

M-F  =  Male-Female 
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Table  2.5 


Correlations  Among  Mean  Predictor  Battery  Performance  Indicator  Scores  Across 
10,152  Combinations  for  Technical/School  Knowledge:  134-164  Minute  Interval 


Indicators 

Val. 

Dis.  V 

BI 

W-B 

W-H 

M-F 

Validity 

1.00 

Discriminant  Validity 

0.88 

1.00 

Brogden  Index 

0.88 

1.00 

1.00 

W-B  Difference 

0.66 

0.55 

0.53 

1.00 

W-H  Difference 

0.51 

0.42 

0.42 

0.69 

1.00 

M-F  Difference 

0.75 

0.87 

0.87 

0.48 

0.45 

1.00 

Table  2,6 

Correlations  Among  Mean  Predictor  Battery  Performance  Indicator  Scores  Across 
6,509  Combinations  for  Technical/School  Knowledge:  194-224  Minute  Interval 


Indicators 

Val. 

Dis.  V 

BI 

W-B 

W-H 

M-F 

Val idity 

1.00 

Discriminant  Validity 

0.92 

1.00 

Brogden  Index 

0.92 

1.00 

1.00 

W-B  Difference 

0.65 

0.57 

.56 

1.00 

W-H  Difference 

0.44 

0.39 

.39 

0.60 

1.00 

M-F  Difference 

0.73 

0.86 

.84 

0.50 

0.48 

1.00 
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Table  2.7 


Correlations  Among  Mean  Predictor  Battery  Performance  Indicator  Scores  Across 
210  Combinations  for  On-the-Job  Core  Technical  Proficiency:  74-104  Minute 
Interval 


Indicators 

Val. 

Dis.  V 

BI 

W-B 

W-H 

M-F 

Validity 

1.00 

Discriminant  Validity 

0.56 

1.00 

Brogden  Index 

0.43 

0.94 

1.00 

W-B  Difference 

0.61 

0.35 

0.17 

1.00 

W-H  Difference 

0.48 

0.09 

-0.04 

0.85 

1.00 

M-F  Difference 

0.57 

0.61 

0.56 

0.44 

0.27 

1.00 

Table  2.8 

Correlations  Among  Mean 
10,152  Combinations  for 
Interval 

Predictor  Battery  Performance  Indicator  Scores  Across 
On-the-Job  Core  Technical  Proficiency:  134-164  Minute 

Indicators 

Val. 

Dis.  V 

BI 

W-B  W-H 

M-F 

Validity 

1.00 

Discriminant  Validity 

0.46 

1.00 

Brogden  Index 

0.41 

0.93 

1. 00 

W-B  Difference 

0.83 

0.36 

0.27 

1.00 

W-H  Difference 

0.38 

0.08 

0.04 

0.61  1.00 

M-F  Difference 

0.44 

0.75 

0.67 

0.45  0.37 

1.00 
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Table  2.9 


Correlations  Among  Mean  Predictor  Battery  Performance  Indicator  Scores  Across 
6,509  Combinations  for  On-the-Job  Core  Technical  Proficiency:  194-224  Minute 
Interval 


Indicators 

Val. 

Dis.  V 

BI 

W-B 

W-H 

M-F 

Validity 

1.00 

Discriminant  Validity 

0.48 

1.00 

Brogden  Index 

0.35 

0.91 

1.00 

W-B  Difference 

0.75 

0.24 

0.12 

1.00 

W-H  Difference 

0.18 

-0.06 

-0.10 

0.49 

1.00 

M-F  Difference 

0.43 

0.73 

0.60 

0.36 

0.36 

1.00 

Table  2.10 

Correlations  Among  Mean  Predictor  Battery  Performance  Indicator  Scores  Across 
210  Combinations  for  On-the-Job  Hands-On  Test  Performance:  74-104  Minute 
Interval 


Indicators 

Val. 

Dis.  V 

BI 

W-B 

W-H 

M-F 

Validity 

1.00 

Discriminant  Validity 

0.67 

1.00 

Brogden  Index 

0.56 

0.92 

1.00 

W-B  Difference 

0.26 

0.33 

0.11 

1.00 

W-H  Difference 

0.21 

0.26 

0.10 

0.88 

1.00 

M-F  Difference 

0.67 

0.71 

0.61 

0.19 

0.04 

1.00 
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Table  2.11 


Correlations  Among  Mean  Predictor  Battery  Performance  Indicator  Scores  Across 
10,152  Combinations  for  On-the-Job  Hands-On  Test  Performance:  134-164  Minute 
Interval 


Indicators 

Val. 

Dis.  V 

BI 

W-B 

W-H 

M-F 

Validity 

1.00 

Discriminant  Validity 

0.57 

1.00 

Brogden  Index 

0.47 

0.91 

1.00 

W-B  Difference 

0.69 

0.32 

0.19 

1.00 

W-H  Difference 

0.37 

0.25 

0.19 

0.65 

1.00 

M-F  Difference 

0.51 

0.63 

0.51 

0.32 

0.24 

1.00 

Table  2.12 

Correlations  Among  Mean 
6,509  Combinations  for 
Interval 

Predictor  Battery  Performance  Indicator  Scores 
On-the-Job  Hands-On  Test  Performance:  194-224 

Across 

Minute 

Indicators 

Val. 

Dis.  V 

BI 

W-B 

W-H 

M-F 

Validity 

1.00 

Discriminant  Validity 

0.58 

1.00 

Brogden  Index 

0.38 

0.92 

1.00 

W-B  Difference 

0.68 

0.22 

0.10 

1.00 

W-H  Difference 

0.27 

0.18 

0.14 

0.50 

1.00 

M-F  Difference 

0.42 

0.49 

0.37 

0.22 

0.21 

1.00 

Battery  Rankings  for  Performance  Indices 

Appendix  A  presents  results  listing  the  top  20  potential  test  batteries 
identified  for  each  of  the  six  test  battery  performance  indices.  These 
results  are  reported  separately  for  each  of  the  nine  sets  of  analyses  (Tables 
A1  through  A9).  That  is,  results  are  given  for  each  time  interval  within  each 
of  the  three  criterion  measures.  Each  table  presents  a  particular 
criterion/time  interval  combination;  each  table  has  six  lists  of  equations 
rank  ordered  by  a  performance  index. 
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The  first  list  (i.e.,  page)  in  each  table  is  the  20  combinations  of 
predictors  with  the  greatest  mean  absolute  validity  across  the  nine  MOS. 

These  20  combinations  are  itemized  under  the  heading  NV  Predictor  Variables. 
Also  shown  for  each  combination  are  the  mean  discriminant  validity,  the 
Brogden  index,  the  mean  White  vs.  Black  difference  estimate,  the  mean  White 
vs.  Hispanic  difference  estimate,  the  Male  vs.  Female  difference  estimate,  and 
the  required  test  administration  time  associated  with  that  combination  of 
tests.  (The  columns  for  these  indices  are  labeled  Validtv  Est  (Mean  and 
S.D. ) ,  Discr  Vldty,  Broq.  Index.  Whte-Blck  (Mean  and  Rnqe).  Whte-Hisp  (Mean 
and  Rnqe) .  Male-Fern  (Mean  and  Rnqe).  and  Test  Time,  respectively.) 

The  second  through  sixth  lists  (i.e.,  pages)  in  each  Appendix  A  table 
follow  the  same  format  to  present,  in  turn,  the  top  20  combinations  of 
predictors  with  respect  to  discriminant  validity,  the  Brogden 's  index,  the 
minimization  of  White  vs.  Black  differences,  the  minimization  of  White  vs. 
Hispanic  differences,  and  the  minimization  of  Male  vs.  Female  differences, 
respectively. 

Note  that  the  last  row  in  each  listing  reports  the  results  associated 
with  a  test  battery  solely  consisting  of  the  ASVAB  Arithmetic  Reasoning  and 
Word  Knowledge  subtests  (labeled  Base) . 


DISCUSSION 

This  section  discusses  the  results  and  addresses  the  issues  associated 
with  identifying  optimal  predictor  batteries  for  each  criterion  measure.  The 
following  questions  direct  the  discussion  for  each  criterion  measure: 

1.  Across  the  three  battery  time  intervals,  what  is  the  average 
battery  value  on  each  of  the  six  indices? 

The  average  value  of  each  index  for  all  the  potential  test  batteries  at 
each  time  interval  is  summarized  in  each  of  the  subsections  titled  "Mean 
Values  on  Battery  Performance  Indices." 

2.  How  do  the  mean  values  for  the  six  indices  change  as  battery 
length  increases? 

We  will  examine  potential  gains  or  trends  in  the  indices  as  the  battery 
length  is  extended  in  the  subsections  titled  "Changes  in  Index  Performance 
With  Increasing  Battery  Length." 

3.  What  trade-offs  are  associated  with  optimizing  performance  on  one 
index  vs.  the  other  indices?  In  other  words,  what  are  the 
correlations  among  the  six  indices? 

The  magnitudes  of  the  relationships  among  the  test  battery  indicators 
are  discussed  in  order  to  illustrate  the  trade-offs  among  indices  that  may  be 
faced  in  situations  where  batteries  must  be  formed  from  a  set  of  available 
subtests.  As  an  example,  one  battery  could  maximize  the  absolute  validity 
criterion  but  not  also  simultaneously  minimize  one  or  more  of  the  specific 
indices  of  subgroup  differences.  This  subsection  is  titled  "Relationships 
Among  Indices"  in  the  discussion  for  each  criterion. 
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4.  Are  certain  subtests  (or  type  of  subtests,  e.g.,  computer- 
administered)  favored  in  the  top  20  rankings  for  a  specific  index 
or  across  a  variety  of  indices?  That  is,  will  forming  a  battery 
by  optimizing  alternative  indices  favor  the  same  or  different 
subtests? 

The  relative  frequencies  with  which  the  various  subtests  are  included  in 
the  listings  of  the  top  20  batteries  are  mentioned  in  the  subsections, 
"Frequencies  of  Subtest  Occurrence  in  Rankings." 

5.  How  do  the  two  indices  of  classification  efficiency  relate  —  do 
they  order  tests  similarly  or  differently? 

We  will  examine  the  degree  of  agreement  between  these  two  indices  in 
terms  of  the  frequency  with  which  each  subtest  is  included  in  the  index 
rankings.  These  comparisons  are  discussed  in  the  subsections  "Discriminant 
Validity  vs.  the  Brogden  Index." 

Predicting  Technical/School  Knowledge 
Mean  Values  on  Battery  Performance  Indices 

The  average  level  of  absolute  validity  (mean  validity  coefficient) 
predicting  Technical/School  Knowledge  across  the  three  time  intervals  is 
substantial  (about  .74  -  .77;  see  Table  2.3).  The  variability  of  the 
validities  is  relatively  small  (i.e.,  the  standard  deviations  are  <  .01).  The 
average  discriminant  validities  range  from  .013  to  .031.  The  average  Brogden 
Index  is  about  .137. 

The  average  estimates  of  White  vs.  Black  differences  are  slightly  less 
than  one  and  a  half  standard  deviations.  The  average  estimates  of  White  vs. 
Hispanic  are  slightly  more  than  a  standard  deviation.  Finally,  the  estimates 
of  Male  vs.  Female  differences  are  between  one-quarter  and  one-third  of  a 
standard  deviation. 

Changes  in  Index  Values  With  Increasing  Battery  Length 

Relatively  small  increases  in  validity,  either  absolute  or  discriminant, 
occur  as  the  length  of  the  battery  is  extended  (see  Table  2.3).  For  example, 
the  mean  validity  coefficient  for  the  prediction  of  Technical/School  Knowledge 
increases  by  about  .03  as  the  battery  length  is  increased  from  the  74-104 
minutes  range  to  the  194-224  minutes  range.  Similarly,  the  increase  in 
discriminant  validity  between  the  shortest  and  longest  time  interval  is  less 
than  .02.  The  Brogden  index  increases  by  about  .06  as  the  battery  time  is 
lengthened.  There  seems  to  be  a  very  slight  increase  in  subgroup  differences 
as  the  time  interval  lengthens;  however,  the  increases  from  the  shortest  to 
the  longest  time  interval  are  less  than  one-tenth  of  a  standard  deviation. 

Relationships  Among  Indices 

Tables  2.4  through  2.6  show  the  relationships  among  the  equation 
performance  indices  for  Technical/School  Knowledge.  There  is  a  substantial 
positive  relationship  between  absolute  validity  and  discriminant  validity 
(.88,  .88,  and  .92  for  the  short,  moderate,  and  long  intervals).  This  means 
that  combinations  of  tests  with  relatively  high  mean  absolute  validities  are 


108 


likely  to  also  have  relatively  high  mean  discriminant  validities.  The 
relationships  between  absolute  validity  and  the  Brogden  index  are  of  the  same 
strength  as  the  absolute  validity  -  discriminant  validity  relationships. 

The  correlations  between  absolute  validity  and  W-B  differences  and 
between  discriminant  validity  and  W-B  differences  are  relatively  substantial 
for  the  three  time  intervals.  This  means  that  combinations  of  subtests  with 
relatively  high  mean  absolute  validities  and  relatively  high  mean  discriminant 
validities  are  very  likely  to  have  relatively  high  levels  of  W-B  differences. 
The  correlations  of  absolute  validity  and  discriminant  validity  with  W-H 
differences  are  lower  than  with  W-B  differences  for  all  three  time  intervals. 

The  relationships  of  absolute  validity  and  discriminant  validity  with 
M-F  differences  are  the  strongest  of  all  (in  the  .78  -  .87  range);  these 
correlations  drop  off  slightly  as  the  time  interval  increases.  An  important 
point  is  that  the  M-F  indices  of  M-F  differences  have  stronger  positive 
relationships  with  discriminant  validity  than  with  absolute  validity.  We  will 
address  this  issue  in  more  detail  later.  The  three  indicators  of  subgroup 
differences  are  positively  related.  However,  another  way  in  which  M-F 
differences  are  unique  is  that  they  are  more  weakly  related  to  both  W-B  and  W- 
H  subgroup  differences  than  the  W-B  and  W-H  indices  are  to  each  other. 

Frequencies  of  Subtest  Occurrence  in  Rankings 

Information  about  predictor  combinations  on  Technical/School  Knowledge 
is  shown  in  Appendix  Tables  Al,  A2,  and  A3  for  the  short,  moderate,  and  long 
time  intervals,  respectively.  For  all  three  intervals,  the  top  20  equations 
in  terms  of  absolute  and  discriminant  validity  all  include  the  ASVAB  subtest 
Auto  and  Shop  Information.  The  computer  tests  are  also  common  in  these 
equations. 

The  spatial  tests  occur  much  more  frequently  in  the  rankings  for 
minimizing  subgroup  differences  than  for  maximizing  either  absolute  validity 
or  discriminant  validity.  Also  for  the  top  20  batteries  that  minimize  each  of 
the  three  types  of  subgroup  differences,  the  ASVAB  subtest  Coding  Speed  shows 
up  consistently  and  Auto  and  Shop  Information  does  not  appear.  At  all  three 
intervals,  the  ASVAB  subtest  Numerical  Operations  consistently  appears  in  the 
batteries  that  minimize  W-B  and  W-H  differences;  at  the  two  longer  intervals. 
Mathematical  Knowledge  also  appears  consistently. 

Discriminant  Validity  vs.  the  Broaden  Index 

Tables  2.4  through  2.6  show  that  there  is  complete  agreement  between  the 
two  indices  of  classification  efficiency.  The  correlations  are  .99  for  the 
shortest  interval  and  1.00  for  the  other  two  intervals.  Although  it  is  no 
surprise  that  they  are  strongly  correlated,  the  magnitude  of  the  relationships 
is  higher  than  expected. 

Predicting  On-the-Job  Core  Technical  Proficiency 
Mean  Values  on  Battery  Performance  Indices 

The  average  level  of  absolute  validity  (mean  validity  coefficient) 
predicting  On-the-Job  Core  Technical  Proficiency  across  the  three  time 
intervals  is  substantial  (about  .59  -  .62,  see  Table  2.3).  The  variability  of 
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the  validities  is  relatively  small  (i.e.,  the  standard  deviations  are  <  .01). 
The  average  discriminant  validities  range  from  .012  to  .027  across  the  three 
time  intervals.  The  average  Brogden  indices  range  from  .123  to  .186. 

The  average  estimates  of  White  vs.  Black  differences  are  slightly  above 
one  and  a  third  standard  deviations.  The  average  estimates  of  White  vs. 
Hispanic  differences  are  approximately  one  standard  deviation.  Finally,  the 
estimates  of  Male  vs.  Female  differences  are  between  three-  and  four-tenths  of 
a  standard  deviation. 

Changes  in  Index  Values  With  Increasing  Battery  Length 

There  are  relatively  small  increases  in  validity,  either  absolute  or 
discriminant,  as  the  length  of  the  battery  is  extended  (see  Table  2.3).  The 
mean  validity  coefficient  for  the  prediction  of  On-the-job  Core  Technical 
Proficiency  increases  by  .03  as  the  battery  length  is  increased  from  the  74- 
104  minute  range  to  the  194-224  minute  range,  and  the  increase  in  discriminant 
validity  between  the  shortest  and  longest  time  interval  is  less  than  .02. 
Again,  as  for  the  Technical/School  Knowledge  criterion,  there  is  an  increase 
of  about  .06  in  the  Brogden  index. 

There  seems  to  be  a  general,  but  very  slight,  increase  in  subgroup 
differences  between  the  first  two  time  intervals  and  then  a  smaller  increase 
in  subgroup  differences  between  the  medium  interval  and  the  long  time 
interval.  The  increase  from  the  short  to  the  medium  time  interval  is  in  no 
case  greater  than  one-  tenth  of  a  standard  deviation. 

Relationships  Among  Indices 

The  relationships  among  the  equation  performance  indices  for  On-the-Job 
Core  Technical  Proficiency  are  shown  in  Tables  2.7  through  2.9.  For  all  three 
time  intervals,  there  is  a  moderate  positive  relationship  between  absolute 
validity  and  discriminant  validity.  This  means  that  combinations  of  tests 
with  relatively  high  mean  absolute  validities  are  likely  to  have  relatively 
high  mean  discriminant  validities.  Correlations  of  absolute  validity  with  the 
Brogden  index  are  slightly  lower  than  with  discriminant  validity  for  all  three 
time  intervals.  Correlations  between  discriminant  validity  and  the  Brogden 
index  range  from  .94  for  the  short  time  interval  to  .91  for  the  longest  time 
interval . 

The  correlations  of  absolute  validity  with  W-B  differences  are 
relatively  substantial.  This  means  that  combinations  of  tests  with  relatively 
high  mean  absolute  validities  are  very  likely  to  have  relatively  high  levels 
of  W-B  differences.  Correlations  between  discriminant  validity  and  W-B 
differences  are  slightly  more  than  half  the  magnitude  of  the  corresponding 
correlations  between  absolute  validity  and  W-B  differences. 

The  relationship  between  absolute  validity  and  W-H  differences  is 
relatively  weaker,  and  drops  off  from  .48  for  the  short  time  interval  to  only 
.18  for  the  longest  time  interval.  The  correlations  between  discriminant 
validity  and  W-H  differences  are  very  low,  indicating  almost  no  relationship 
between  these  indices. 

The  relationship  between  absolute  validity  and  M-F  differences  is 
moderate  (ranging  from  .  57  for  the  shortest  to  .43  for  the  longest  interval). 
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However,  the  relationship  between  discriminant  validity  and  M-F  differences  is 
more  substantial  (approximately  three-tenths  of  a  standard  deviation  at  the 
medium  and  longest  time  intervals).  The  three  indices  of  subgroup  differences 
are  positively  related:  W-B  and  W-H  are  more  strongly  related  to  each  other 
than  either  is  to  M-F  differences,  with  the  lowest  correlations  between  W-H 
and  M-F, over  the  three  time  intervals. 

Frequencies  of  Subtest  Occurrence  in  Ranking 

Information  about  predictor  combinations  of  On-the-Job  Core  Technical 
Proficiency  is  shown  in  Appendix  Tables  A4,  A5,  and  A6  for  the  74-104,  134- 
164,  and  194-224  minute  time  intervals,  respectively.  The  top  20  battery 
rankings  for  absolute  validity  and  discriminant  validity  all  include  (with  two 
exceptions  in  the  shortest  interval)  the  ASVAB  subtest  Auto  and  Shop 
Information.  At  the  two  longer  time  intervals  the  ASVAB  subtest  Coding  Speed 
is  also  in  every  equation.  The  Short-Term  Memory  computer  subtest  is  also 
common  in  these  equations.  In  the  two  longer  time  intervals,  the  spatial 
tests  appear  more  frequently  in  the  validity  battery  rankings,  whereas  the 
computer  tests  occur  more  frequently  in  the  discriminant  validity  battery 
rankings. 

The  Numerical  Operations  test  does  not  show  up  in  the  rankings  for 
validity,  discriminant  validity,  or  minimizing  M-F  differences,  but  does  show 
up  consistently  in  the  rankings  for  minimizing  both  W-B  and  W-H  differences. 
Auto  and  Shop  Information  does  not  appear  in  the  rankings  for  minimizing  each 
of  the  three  types  of  adverse  impact,  but  the  ASVAB  subtests  Coding  Speed  and 
Mathematical  Knowledge  do  show  up  relatively  frequently.  The  difference 
between  the  batteries  that  minimize  W-B  and  W-H  differences  and  the  batteries 
that  minimize  M-F  differences  is  that  the  former  include  a  fair  number  of 
computer  tests  but  the  latter  include  very  few  computer  tests. 

Discriminant  Validity  vs.  the  Broqden  Index 

As  shown  in  Tables  2.7  through  2.9,  the  Brogden  index  and  discriminant 
validity  are  again  very  strongly  related,  but  not  to  the  same  degree  as  for 
the  Technical/School  Knowledge  criterion.  For  the  shortest  interval,  the 
correlation  is  .94;  this  decreases  to  ,93  and  .91  as  the  time  interval 
lengthens. 


Predicting  On-the-Job  Hands-On  Test  Performance 
Mean  Values  on  Battery  Performance  Indices 

The  average  level  of  absolute  validity  (mean  validity  coefficient) 
predicting  On-the-Job  Hands-On  Test  Performance  across  the  three  time 
intervals  is  relatively  moderate  (about  .46  -  .50;  see  Table  2.3).  The 
variability  of  the  validities  is  relatively  small  (i.e.,  the  standard 
deviations  are  <  .02).  The  average  discriminant  validities  range  from  .017  to 
.030.  The  average  Brogden  indices  range  from  .132  to  ,187. 

The  average  estimates  of  White  vs.  Black  differences  are  approximately 
one  and  a  fifth  standard  deviations.  The  average  estimates  of  White  vs. 
Hispanic  differences  are  around  nine-tenths  of  a  standard  deviation.  Finally, 
the  estimates  of  Male  vs.  Female  differences  are  between  five-tenths  and  two- 
thirds  of  a  standard  deviation. 
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Changes  in  Index  Values  With  Increasing  Battery  Length 

As  shown  in  Table  2.3,  there  are  relatively  small  increases  in  validity, 
either  absolute  or  discriminant,  as  the  length  of  the  battery  is  extended. 

The  mean  validity  coefficient  for  the  prediction  of  On-the-Job  Hands-On  Test 
Performance  increases  by  about  .04  as  the  time  interval  is  increased  from  the 
shortest  to  the  longest  battery  length.  Once  again,  the  increase  in 
discriminant  validity  is  less  than  .02,  and  the  increase  in  the  Brogden  index 
between  the  shortest  and  longest  time  interval  is  about  .06.  There  is  very 
little  change  in  subgroup  differences  over  the  three  time  intervals  for  the  W- 
B  and  W-H  indices,  but  a  more  marked  increase  of  .13  standard  deviation  over 
these  intervals  for  the  M-F  index. 

Relationships  Among  Indices 

The  relationships  among  the  equation  performance  indices  for  On-the-Job 
Hands-On  Test  Performance  are  shown  in  Tables  2.10  through  2.12.  There  is  a 
relatively  moderate  positive  relationship  between  absolute  validity  and 
discriminant  validity.  This  means  that  combinations  of  tests  with  relatively 
high  mean  absolute  validities  are  likely  to  have  relatively  high  mean 
discriminant  validities.  This  holds  for  all  three  time  intervals.  In 
comparison,  absolute  validity  correlates  less  strongly  with  the  Brogden  index. 
The  correlations  between  discriminant  validity  and  the  Brogden  index  are  .92, 
.91,  and  .92  for  the  short,  moderate,  and  long  intervals,  respectively. 

The  correlations  between  absolute  validity  and  W-B  differences  vary 
greatly  --  from  .26  to  .69  and  .68  as  the  battery  length  increases.  For  the 
longer  two  intervals  only,  this  means  that  combinations  of  tests  with 
relatively  high  mean  absolute  validities  are  very  likely  to  have  relatively 
high  levels  of  W-B  differences.  This  means  that  the  chances  of  discovering 
combinations  of  tests  with  relatively  low  levels  of  W-B  differeances  and 
relatively  high  absolute  validities  are  better  for  the  shorter  time  interval. 
The  correlations  between  absolute  validity  and  W-H  differences  and 
discriminant  validity  and  W-H  differences  are  relatively  low. 

The  relationships  between  absolute  validity  and  M-F  differences  and 
discriminant  validity  and  M-F  differences  for  the  shortest  interval  are 
moderate  (.67  and  .71,  respectively)  but  decrease  as  the  time  interval 
increases.  Again,  the  index  of  M-F  differences  is  the  only  indicator  of 
subgroup  differences  that  has  a  stronger  positive  relationship  with 
discriminant  validity  than  with  absolute  validity.  Again,  the  three  indices 
of  subgroup  differences  are  positively  related;  however,  W-B  and  W-H 
differences  are  more  related  to  each  other  than  either  is  to  M-F  differences. 

Freguencies  of  Subtest  Occurrence  in  Rankings 

Appendix  Tables  A7,  A8,  and  A9  contain  the  battery  rankings  for 
predicting  On-the-Job  Hands-On  Test  Performance  for  the  short,  moderate,  and 
long  time  intervals,  respectively.  As  for  the  two  criteria  discussed 
previously,  the  absolute  and  discriminant  validity  rankings  all  include  the 
ASVAB  subtest  Auto  and  Shop  Information.  The  computer  tests  are  also  common 
in  these  equations  (especially  Target  Identification,  One-Hand  Tracking,  and 
Two-Hand  Tracking).  Also,  the  computer  tests  show  up  more  frequently  than  the 
spatial  tests  in  the  top  20  absolute  and  discriminant  validity  equations. 
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Auto  and  Shop  Information  does  not  appear  in  the  top  20  subtest 
combinations  that  minimize  each  of  the  three  types  of  subgroup  differences, 
but  the  ASVAB  subtests  Coding  Speed  and  Mathematical  Knowledge  show  up 
relatively  often.  Again,  the  test  batteries  that  minimize  W-B  and  W-H 
differences  include  the  Numerical  Operations  subtest;  this  subtest  is  missing 
from  the  rankings  for  minimizing  M-F  differences  at  the  two  shorter  intervals 
but  does  appear  in  about  half  of  the  top  20  batteries  for  the  longest  time 
interval.  The  batteries  that  minimize  both  W-B  and  W-H  differences  also 
include  a  moderate  number  of  computer  tests;  the  batteries  that  minimize  M-F 
differences  favor  the  spatial  tests  over  the  computer  tests. 

Discriminant  Validity  vs.  the  Broaden  Index 

For  On-the-Job  Hands-On  Test  Performance,  correlations  between  the 
Brogden  index  and  discriminant  validity  are  .92,  .91,  and  .92  for  the  74-104, 
134-164,  and  194-224  minute  intervals,  respectively.  These  correlations  are 
of  the  same  magnitude  as  those  for  the  other  on-the-job  criterion  of  Core 
Technical  Proficiency. 

Classification  Efficiency:  Discriminant  Validity  vs.  the  Brogden  Index 

As  described  in  the  sections  for  each  criterion  measure  above,  there  is 
a  considerable  overlap  between  discriminant  validity  and  the  Brogden  index  of 
classification  efficiency.  It  is  not  a  surprise  that  the  correlations  for  all 
criteria  and  time  interval  combinations  are  above  .90  because  both  indices  are 
measures  of  classification  efficiency.  These  results  indicate  that  the  two 
indices  order  the  potential  test  batteries  very  similarly,  especially  for  the 
Technical/School  Knowledge  criterion.  A  less  predictable  result  is  that  for 
the  Technical/School  Knowledge  criterion,  the  correlations  between  absolute 
validity  and  the  Brogden  index  are  essentially  equivalent  to  those  between 
absolute  validity  and  discriminant  validity,  but  for  the  two  On-the-Job 
criteria  the  correlations  of  absolute  validity  with  the  Brogden  index  are 
lower  than  correlations  of  absolute  validity  with  discriminant  validity. 

Tables  A1  through  A9  in  Appendix  A  show  that  there  is  substantial 
agreement  between  the  two  indices  of  potential  classification  efficiency  in 
terms  of  the  frequency  with  which  each  subtest  is  included  in  each  index's 
list  of  top  20  test  batteries.  There  is,  however,  a  consistent  difference 
between  the  rankings  for  these  two  indices  across  all  tables.  On  the  average, 
more  variables  are  listed  for  each  of  the  equations  for  the  Brogden  index  than 
for  each  of  the  equations  for  discriminant  validity.  Despite  this  difference, 
the  two  indices  function  very  similarly.  Consequently  for  the  remainder  of 
this  chapter,  we  will  focus  on  the  index  of  discriminant  validity. 

"Optimal"  Test  Batteries 

Any  number  of  potential  rules  could  be  used  to  try  to  create  an 
"optimal"  test  battery  that  somehow  considers  all  the  discussed  indices  of 
battery  performance.  As  a  demonstration,  two  such  rules  were  implemented 
using  the  "top  twenty"  potential  test  batteries  in  Appendix  A. 
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Two  Potential  "Rules"  for  Identifying  Battery  Subtests 


The  first  "rule"  implemented  was  "Maximize  Validity/Minimize  Average 
Subgroup  Differences."  According  to  this  rule,  the  optimal  combination  of 
tests  for  each  criterion  and  time  interval  was  that  battery  of  tests  in  the 
top  20  mean  absolute  validities  list  (see  Appendix  A)  that  also  had  the  lowest 
average  across  the  three  types  of  subgroup  differences.  (This  rule  considered 
only  the  top  20  absolute  validity  equations.)  The  optimal  equations  according 
to  this  rule  for  each  criterion  and  time  interval  are  presented  in  Table  2.13. 

According  to  these  optimal  equations,  there  is  some  improvement  in 
absolute  validity,  discriminant  validity,  and  the  Brogden  index  between  the 
short  time  interval  and  the  medium  time  interval;  however,  the  improvement  in 
most  cases  is  less  dramatic  between  the  medium  time  interval  and  the  long  time 
interval . 

The  mean  W-B  difference  ranges  from  1.2  to  1.4  standard  deviations  for 
all  the  criterion/time  interval  combinations.  The  W-H  difference  is 
approximately  one  standard  deviation  for  Technical/School  Knowledge  and  On- 
the-Job  Core  Technical  Proficiency,  but  appears  to  be  somewhat  smaller  for  On- 
the-Job  Hands-On  Test  Performance.  The  M-F  difference  is  less  than  one-half 
of  a  standard  deviation  for  Technical/School  Knowledge  and  On-the-Job  Core 
Technical  Proficiency,  but  is  substantially  greater  for  On-the-Job  Hands-On 
Test  Performance. 

Beyond  the  automatically  included  Arithmetic  Reasoning  and  Word 
Knowledge,  Auto  and  Shop  Information  is  in  almost  every  optimal  equation. 

The  computer-administered  memory  test  shows  up  in  all  the  longer  batteries. 

The  computer-administered  tracking  tests  appear  in  the  longest  battery  for 
School  Knowledge  and  the  intermediate-length  battery  for  Hands-On,  not  at  all 
for  Core  Technical  Performance. 

The  second  "rule"  implemented  was  "Maximize  Discriminant  Validity/ 
Minimize  Average  Subgroup  Difference."  According  to  this  rule,  the  optimal 
subtest  combination  for  each  criterion  and  time  interval  was  that  battery  of 
tests  in  the  top  20  mean  discriminant  validities  list  (see  Appendix  A)  that 
also  had  the  lowest  average  across  the  three  types  of  subgroup  differences. 
(Again,  this  rule  considered  only  the  top  20  batteries  ranked  according  to  the 
discriminant  validity  index.)  The  optimal  equations  according  to  this  rule 
for  each  criterion  and  time  interval  are  presented  in  Table  2.14. 

The  results  in  Table  2.14  are  in  most  respects  very  similar  to  the 
results  in  Table  2.13.  A  major  difference  is  that  the  computer  tests  occur 
more  often  in  these  "optimal"  equations,  especially  for  the  longer  intervals 
and  for  those  equations  predicting  On-the-Job  Hands-On  Test  Performance.  This 
result  suggests  that  the  computer  tests  contribute  more  to  maximizing 
discriminant  validity.  The  problem  is,  however,  that  they  also  appear  to 
contribute  to  M-F  differences.  There  is  a  general  tendency  for  the  estimates 
of  M-F  differeances  in  Table  2.14  to  be  of  greater  magnitude  than  the 
corresponding  estimates  in  Table  2.13.  This  trade-off  between  discriminant 
validity  and  M-F  differences  is  also  demonstrated  by  the  substantial 
correlations  between  discriminant  validity  and  M-F  differences  in  Tables  2.4 
through  2.12  (especially  Tables  2.10,  2.11,  and  2.12). 
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"Optimal"  Test  Batteries  for  Each  Criterion  and  Time  Interval  According  to  the  "Maximize  Discriminant 
Validity/Minimize  Average  Subgroup  Differences"  Rule _ _ 
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Hispanic  Difference;  M-F  =  Male  -  Female  Difference;  T  =  Testing  Time  for  Battery. 
®  See  Table  2.2  for  the  titles  and  sources  of  the  individual  tests. 


Contributions  of  the  ASVAB  Subtests 


Validity  and  subgroup  differences  were  evaluated  at  the  test  battery 
level  because  the  influence  that  a  particular  subtest  has  on  the  test  battery 
performance  indices  depends  on  the  other  subtests  in  that  particular  battery. 
However,  it  is  still  useful  to  observe  the  frequency  with  which  subtests  occur 
in  the  top  20  lists  according  to  each  test  battery  index.  These  observations 
afford  a  general  sense  of  the  tendency  for  a  particular  subtest  to  optimize  a 
test  battery's  contribution  with  respect  to  particular  indices. 

This  section  summarizes  the  subtests  selected  for  the  134-164  minute 
time  interval.  This  is  the  time  interval  that  allows  the  greatest 
differential  inclusion  of  subtests  across  batteries.  No  reference  is  made  to 
either  the  Arithmetic  Reasoning  or  Word  Knowledge  subtests  of  the  ASVAB 
because  these  were  designated  as  the  base  condition  (automatically  included  in 
every  potential  test  battery). 

Auto  and  Shop  Information  contributes  to  absolute  validity  and 
discriminant  validity,  but  not  to  minimizing  all  three  types  of  subgroup 
differences.  Coding  Speed  contributes  to  minimizing  all  three  types  of 
subgroup  differences.  Electronic  Information  tends  to  make  less  of  a 
contribution  to  absolute  validity  and  more  of  a  contribution  to  discriminant 
validity.  General  Science  tends  to  make  more  of  a  contribution  to  absolute 
validity  and  much  less  of  a  contribution  to  discriminant  validity,  and 
Mechanical  Comprehension  tends  not  to  contribute  to  either  absolute  or 
discriminant  validity. 

Mathematical  Knowledge  tends  to  contribute  to  absolute  validity  to  a 
lesser  extent  and  to  discriminant  validity  to  a  greater  extent.  Mathematical 
Knowledge  also  has  a  tendency  to  contribute  to  minimizing  W-B  differences  but 
has  a  greater  tendency  to  contribute  to  minimizing  W-H  and  M-F  differences. 
Numerical  Operations  has  (a)  a  low  tendency  to  contribute  to  absolute  validity 
and  discriminant  validity,  (b)  a  high  tendency  to  contribute  to  minimizing  W-B 
and  W-H  differences,  and  (c)  a  low  tendency  to  contribute  to  minimizing  M-F 
differences.  Paragraph  Comprehension  has  a  high  tendency  to  contribute  to 
minimizing  W-B  and  M-F  differences,  and  a  low  tendency  to  contribute  to 
minimizing  W-H  differences. 

Adding  Subtests  to  the  ASVAB 

Another  question  is  "What  is  gained  by  adding  subtests  to  the  existing 
set  of  ASVAB  subtests?"  Assembling  Objects  tends  to  contribute  to  both 
absolute  validity  and  the  minimization  of  M-F  differences.  Orientation  tends 
not  to  contribute  to  any  indices,  and  Reasoning  tends  to  contribute  to 
minimizing  M-F  differences. 

Target  Identification  has  a  strong  tendency  to  contribute  to 
discriminant  validity  and  a  moderate  tendency  to  contribute  to  minimizing  all 
three  types  of  differences.  One-Hand  Tracking  has  a  low  to  moderate  tendency 
to  contribute  to  absolute  validity  and  a  high  tendency  to  contribute  to 
discriminant  validity.  One-Hand  Tracking  tends  to  contribute  more  to  absolute 
validity  and  discriminant  validity  than  Two-Hand  Tracking  does.  One-Hand  and 
Two-Hand  Tracking  have  a  moderate  tendency  to  contribute  to  minimizing  W-B  and 
W-H  differences  but  not  to  minimizing  M-F  differences. 
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Substituting  Subtests  for  the  ASVAB  Subtests 


The  issue  of  whether  Project  A/Career  Force  subtests  might  be 
substituted  for  one  or  more  of  the  ASVAB  subtests  can  be  approached  from 
several  different  perspectives,  such  as  time  allotted  for  test  administration, 
and  feasibility  of  administering  different  types  of  tests  (e.g.,  computer 
capabilities),  as  well  as  maximizing  validity  indices  or  minimizing  subgroup 
differences . 

The  194-224  minute  interval  most  closely  approximates  the  length  of  the 
current  ASVAB.  For  this  interval,  for  the  Technical/School  Knowledge 
criterion.  Appendix  Table  A3  shows  that  all  ASVAB  subtests  except  Mechanical 
Comprehension,  Coding  Speed,  and  Numerical  Operations  always  appear  in  the 
rankings  for  absolute  validity  and  discriminant  validity.  This  means  that  a 
number  of  spatial  and  computer  tests  tended  to  contribute  more  to  maximizing 
these  two  indices  than  did  these  three  ASVAB  tests.  Therefore,  only  seven  or 
eight  of  the  ASVAB  subtests  (including  Arithmetic  Reasoning  and  Word 
Knowledge),  rather  than  all  ten,  were  included  in  the  batteries  that  included 
12  to  14  subtests.  On  the  other  hand,  for  the  rankings  according  to  subgroup 
differences,  the  subtests  that  were  in  almost  all  cases  excluded  from  the 
batteries  were  Auto  and  Shop  Information,  Electronic  Information,  and 
Mechanical  Comprehension.  These  subtests  were  left  out  of  the  equations 
because  spatial  and  computer  subtests  contributed  more  to  minimizing  subgroup 
differences. 

For  the  194-224  minute  interval  for  the  On-the-Job  Core  Technical 
Proficiency  criterion,  the  subtests  that  appeared  less  frequently  for  the 
absolute  and  discriminant  validity  indices  were  Numerical  Operations  and 
General  Science  (see  Table  A6).  For  the  three  indices  of  subgroup 
differences,  the  subtests  Auto  and  Shop  Information,  Electronic  Information, 
and  Mechanical  Comprehension  failed  to  appear  in  the  top  20  batteries. 

The  results  obtained  for  the  134-164  minute  interval  are  applicable  to 
situations  where  less  testing  time  would  be  available.  For  this  interval  for 
the  Technical/School  Knowledge  criterion  (see  Table  A2),  the  Coding  Speed, 
Mechanical  Comprehension,  and  Numerical  Operations  subtests  were  included 
least  often  of  all  ASVAB  subtests  in  the  battery  rankings  for  the  absolute  and 
discriminant  validity  indices.  Again  as  for  the  longest  interval,  the  Auto 
and  Shop  Information,  Electronic  Information,  and  Mechanical  Comprehension 
subtests  as  well  as  the  General  Science  subtest  were  excluded  from  the  top  20 
rankings  for  the  three  indices  of  subgroup  differences. 

For  the  On-the-Job  Core  Technical  Proficiency  criterion  at  the  134-164 
minute  level  (see  Table  A5),  the  General  Science,  Mechanical  Knowledge,  and 
Numerical  Operations  subtests  were  the  most  often  excluded  in  the  rankings  for 
absolute  and  discriminant  validity.  For  all  three  indices  of  subgroup 
differences,  four  subtests  were  excluded  from  the  rankings:  Auto  and  Shop 
Information,  Electronic  Information,  Mechanical  Comprehension,  and  General 
Science. 

At  the  74-104  minute  interval,  so  few  subtests  are  included  in  the 
batteries  that  it  is  not  very  informative  to  consider  which  ASVAB  subtests  do 
not  occur  in  the  rankings.  Of  course,  the  patterns  mentioned  above  for  the 
longer  intervals  also  hold  for  the  shortest  interval,  and  are  magnified  due  to 
the  time  constraint. 
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Assuming  a  situation  where  ASVAB  subtests  not  included  in  the  rankings 
were  to  be  dropped  from  an  operational  battery,  then  testing  time  could  be 
filled  with  subtests  from  the  Project  A/Career  Force  battery  that  did 
contribute  to  the  indices.  These  subtests  were  described,  corresponding  to 
the  indices  they  contributed  to,  in  the  section  "Adding  Subtests  to  the 
ASVAB,"  above. 


Comparisons  Across  Criteria 

The  validity  coefficients  presented  to  this  point  were  not  corrected  for 
criterion  unreliability.  This  correction  is  necessary  to  make  comparisons 
across  criteria.  Although  the  content  of  the  criterion  measures  differs, 
these  measures  were  developed  and  administered  in  very  similar  ways  across 
MOS.  For  purposes  of  approximation,  we  corrected  the  mean  validity 
coefficients  in  Table  2.3  using  the  criterion  reliability  estimates  listed  in 
Table  2.15.  Table  2.15  provides  the  mean  validity  coefficients  corrected  for 
unreliability.  These  results  are  consistent  with  past  research  --  when  the 
predictors  are  ability  measures,  on-the-job  performance  criteria  (i.e.,  Core 
Technical  Proficiency  and  Hands-On  Test  Performance)  tend  to  be  somewhat  less 
predictable  than  end-of-training  performance  criteria  (i.e.,  Technical/School 
Knowledge) . 


Table  2.15 


Mean  Validity  Coefficients  Corrected  for  Unreliability  for  Each  Criterion  and 
Time  Interval 


Criterion 

D  ^ 

^yy 

74-104 

Mins. 

134-164 

Mins. 

194-224 

Mins. 

Technical  School/Knowledge 

.90 

.779 

.802 

.812 

On-the-Job  Core  Technical 
Proficiency 

.85 

.638 

.662 

.671 

On-the-Job  Hands-On  Test 
Performance 

.50 

.644 

.693 

.707 

^  Reliability  estimates  for  each  criterion  are  based  on  data  from  Campbell 
(1987);  estimates  are  slightly  higher  than  observed  values  so  that 
conservative  corrections  can  be  made. 


Concluding  Remarks 

The  method  of  analysis  we  used  (i.e.,  calculating  the  test  battery 
performance  indices  for  every  possible  combination  of  subtests  at  predeter¬ 
mined  time  intervals)  does  not  identify  the  combination  of  subtests  that  will 
simultaneously  optimize  all  the  test  battery  performance  parameters  considered 
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in  this  report.^  The  results  of  these  analyses  do,  however,  convincingly 
make  the  point  that  such  a  test  battery  does  not  exist.  Examination  of  the 
test  batteries  ordered  by  each  of  the  test  battery  performance  indices  makes 
some  trade-offs  apparent. 

The  more  dramatic  trade-offs  are  (a)  the  maximization  of  absolute 
validity  versus  the  minimization  of  all  three  types  of  subgroup  differences, 
(b)  the  maximization  of  classification  efficiency  versus  the  minimization  of 
all  three  types  of  subgroup  differences,  (c)  the  minimization  of  W-B 
differences  versus  the  minimization  of  M-F  differences,  and  (d)  the 
minimization  of  W-H  differences  versus  the  minimization  of  M-F  differences. 

Predictably,  subtests  in  the  ASVAB  battery  show  a  substantial  tendency 
to  contribute  to  absolute  validity  and  a  moderate  tendency  to  contribute  to 
potential  classification  efficiency.  Compared  to  the  ASVAB  subtests,  the 
Project  A/Career  Force  subtests  show  a  relatively  more  substantial  tendency  to 
contribute  to  classification  efficiency  and  a  relatively  moderate  tendency  to 
contribute  to  absolute  validity.  The  Project  A/Career  Force  subtests  do  not 
seem  to  show  a  dramatically  greater  tendency,  compared  to  the  ASVAB  subtests, 
to  contribute  to  minimizing  the  three  types  of  subgroup  differences. 

It  is  important  to  note  that  there  are  a  number  of  reasons  why  a 
particular  subtest  can  be  included  in  a  top  20  test  batteries  list  according 
to  a  particular  test  battery  performance  index.  First,  all  the  test  batteries 
examined  here  automatically  included  Arithmetic  Reasoning  and  Word  Knowledge; 
therefore,  the  contributions  of  the  other  subtests  to  optimizing  each  index 
were  relative  to  these  two  subtests.  The  reason  is  that  the  contribution  of 
every  subtest  to  the  optimization  of  each  index  is  relative  to  the  other 
subtests  in  that  particular  potential  test  battery. 

The  reasons  for  inclusion  also  depend  on  the  index.  If  the  index  is 
mean  absolute  validity,  a  subtest  that  accounts  for  a  unique  portion  of  the 
variance  in  the  criterion  across  all  jobs  will  appear  in  many  of  the  top  20 
test  batteries.  If  the  index  is  discriminant  validity,  a  subtest  that 
accounts  for  a  unique  portion  of  the  variance  in  the  criterion  in  one  or  more 
of  the  jobs  but  not  in  other  jobs  will  appear  in  many  of  the  top  20  test 
batteries. 

Subtests  can  also  appear  in  the  top  20  test  batteries  ranked  according 
to  the  minimization  of  a  particular  type  of  subgroup  difference  (e.g.,  W-B 
differences)  for  multiple  reasons.  One  way  for  a  subtest  to  contribute  to  a 
relatively  small  subgroup  difference  on  the  predicted  performance  score  is  for 
it  to  predict  a  portion  of  the  variance  in  the  criterion  on  which  there  is  a 
relatively  small  subgroup  difference.  However,  a  subtest's  contribution  to  a 
relatively  small  subgroup  difference  on  the  predicted  performance  score  may  be 
independent  of  the  subtest's  validity  or  the  size  of  the  subgroup  difference 
on  that  subtest.  This  is  because  the  subtest's  contribution  to  the  predicted 
performance  score  is  a  function  of  the  regression  weight  that  is  assigned  to 
it  by  the  least-squares  regression  equation. 


'  This  method  was  employed  with  a  set  of  tests  that  had  been  earefully  chosen  and  for  which  criterion- 
related  validity  had  already  been  demonstrated;  we  do  not  recommend  this  technique  for  purely  exploratory 
validation  analyses. 
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The  point  is  that  a  subtest  that  receives  a  small  regression  weight,  for 
whatever  reason,  will  have  very  little  influence  on  the  predicted  performance 
score  and  will  not  increase  the  subgroup  difference  on  the  score  substan¬ 
tially.  Therefore,  a  subtest  might  be  included  in  a  potential  test  battery 
that  has  a  relatively  small  subgroup  differences  because  that  subtest  has  a 
small  influence  on  the  predicted  performance  score  but  helps  the  battery  reach 
the  predetermined  time  interval. 

As  an  example,  assume  that:  (a)  two  subtests  require  the  same  amount  of 
time  to  administer,  (b)  the  first  subtest,  upon  which  Males  score  higher  than 
Females,  would  increase  the  absolute  validity  of  the  test  battery 
substantially,  and  (c)  the  second  subtest  is  unreliable  and  not  related  to  the 
criterion,  but  Males  and  Females  do  not  score  differently  on  it.  All  else 
being  equal,  the  minimization  of  M-F  differences  will  favor  a  battery  that 
includes  the  second  subtest  over  a  battery  that  includes  the  first  subtest. 

An  important  point  is  that  we  imposed  the  condition  that  each  test 
battery  fit  into  one  of  three  predetermined  administration  time  intervals. 
Therefore,  a  combination  of  subtests  may  have  ranked  in  the  top  20  test 
batteries  for  a  particular  index  not  because  it  performed  well  on  that  index, 
but  because  it  performed  well  relative  to  the  other  potential  test  batteries 
that  also  fit  into  that  time  interval. 

A  Suggested  Approach  —  Conducting  Utility  Analyses  for  a  Narrowed  Set  of 
Batteries 


In  terms  of  evaluating  the  value  of  a  particular  test  battery  utility, 
mean  absolute  validity  and  discriminant  validity  are  good  indices  for  ordering 
equations  within  a  time  interval;  however,  mean  absolute  validity  and 
discriminant  validity  may  not  provide  the  best  possible  information  about  the 
relative  utility  of  shorter  versus  longer  batteries.  To  make  this  type  of 
inference,  the  approach  described  here  could  be  combined  with  the  MPP  approach 
proposed  by  Johnson  &  Zeidner  (1990).  The  steps  would  be  to  (a)  identify  some 
number  of  top  equations  for  each  time  interval,  for  each  of  the  indices  we 
have  used  in  this  chapter,  and  (b)  calculate  MPP  using  computer  simulations  to 
generate  an  estimate  of  the  potential  job  performance  gain  associated  with 
specific  test  batteries. 

Implications  of  Weighing  Alternative  Goals  in  Complying  With  Professional  and 
Legal  Guidelines 

Value  judgments  will  be  faced  in  any  situation  where  a  battery  of 
subtests  is  formed  for  operational  use.  We  have  described  some  of  the  indices 
to  consider  in  these  situations.  If  judgments  regarding  the  relative  priority 
of  optimizing  each  of  the  test  battery  parameters  are  not  made  explicitly, 
they  will  have  been  made  implicitly  by  default. 

The  results  reported  here  have  important  implications  for  organizations 
attempting  to  comply  with  professional  and  legal  guidelines  regarding  test 
development  and  use.  The  Uniform  Guidelines  stipulate  that  employers  search 
for  alternatives  among  available  predictors  that  are  job-related;  these 
alternatives  must  at  the  same  time  have  less  adverse  impact  against  protected 
subgroups.  Our  analyses  suggest  that  the  subtests  included  in  batteries 
selected  to  minimize  differences  among  subgroups  are  not  equivalent  to  those 
included  in  batteries  selected  to  maximize  absolute  (or  discriminant) 
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validity.  They  further  indicate  that  the  subtests  included  in  batteries 
selected  to  minimize  differences  between  one  set  of  subgroups  often  differ 
substantially  from  the  subtests  included  in  batteries  selected  to  minimize 
differences  between  a  different  set  of  subgroups. 
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Chapter  3 

BASIC  VALIDATION  RESULTS  FOR  THE  LVII  SAMPLE 
Scott  H.  Oppler,  Norman  6.  Peterson,  and  Andrew  M.  Rose 


This  chapter  summarizes  the  results  of  the  validity  evaluation  of  the 
ASVAB  and  the  Project  A  Longitudinal  Validation  (LV)  Experimental  Battery  for 
predicting  second-tour  performance  in  the  Army.  The  results  are  based  on  the 
second-tour  performance  data  collected  from  the  Project  A/Career  Force 
longitudinal  sample.  They  extend  the  val idation 'results  reported  previously 
(Oppler,  Peterson,  &  Russell,  1994)  for  first-tour  (LVI)  performance. 

OBJECTIVES 

The  objectives  of  the  analyses  described  in  this  chapter  are  as  follows: 

(1)  Compute  the  basic  validities  for  ASVAB  and  Experimental  Battery 
predictors  against  the  second-tour  performance  factors  and 
selected  individual  performance  measures. 

(2)  Compare  the  validities  of  four  alternative  sets  of  ASVAB  scores 
(nine  ASVAB  subtests  vs.  four  ASVAB  factors  vs.  AFQT  vs.  MOS- 
appropriate  Aptitude  Area  composites). 

(3)  Compare  the  validities  of  three  alternative  sets  of  ABLE  scores. 

(4)  Assess  the  incremental  validities  for  the  Experimental  Battery 
predictors  over  the  four  ASVAB  factor  composites. 

(5)  Compare  the  incremental  validities  of  three  alternative  sets  of 
ABLE  scores. 

(6)  Compare  the  validities  and  incremental  validities  of  the 
Experimental  Battery  predictors  reported  for  LVI  with  the 
validities  and  incremental  validities  reported  for  LVII. 

PROCEDURES 

Sample 

The  results  reported  here  are  based  on  a  different  strategy  for  editing 
the  LVII  sample  than  was  previously  used  for  the  LVI  analyses.  To  review 
briefly,  the  LVI  analyses  used  two  sampling  plans,  a  "listwise  deletion"  plan 
and  a  "setwise  deletion"  plan.  The  first  of  these  strategies  mirrored  the 
procedure  used  to  evaluate  the  Project  A  CVI  predictors  against  first-tour 
performance.  To  be  included  in  those  analyses,  soldiers  in  the  CVI  sample 
were  reguired  to  have  complete  data  for  all  of  the  Project  A  CV  predictor 
composites,  a^  well  as  for  the  ASVAB  and  all  of  the  CVI  first-tour  performance 
factors.  Corresponding  to  this  CV  strategy,  a  validation  sample  composed 
solely  of  soldiers  having  complete  data  for  all  the  LVI  Experimental  Battery 
predictors,  the  ASVAB,  and  the  LV  first-tour  performance  factors  was  created^^ 
for  the  LVI  data  set.  This  sample  was  referred  to  as  the  "listwise  deletion" 
sample. 
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In  the  alternative  LVI  sample  editing  strategy,  a  separate  validation 
sample  was  identified  for  each  set  of  predictors  in  the  Experimental  Battery. 
More  specifically,  to  be  included  in  the  validation  sample  for  a  given 
predictor  set,  soldiers  were  required  to  have  complete  data  for  each  of  the 
first-tour  performance  factors,  the  ASVAB,  and  the  predictor  composites  in 
that  predictor  set  only.  For  example,  a  soldier  who  had  data  for  the  complete 
set  of  ABLE  composites  (as  well  as  complete  ASVAB  and  criterion  data),  but  was 
missing  data  from  the  AVOICE  composites,  would  have  been  included  in  the 
"setwise  deletion"  sample  for  estimating  the  validity  of  the  (ABLE)  test,  but 
not  in  the  sample  for  the  AVOICE  test. 

The  principal  reason  for  creating  these  setwise  deletion  samples  was  to 
maximize  the  sample  sizes  used  to  estimate  the  validity  of  the  Experimental 
Battery  predictors.  The  setwise  sample  sizes  were,  in  fact,  considerably 
larger  than  those  associated  with  the  listwise  strategy. 

The  number  of  soldiers  with  complete  predictor  and  criterion  data  (the 
listwise  deletion  sample)  in  each  MOS  is  reported  in  Table  3.1  for  both  the 
LVI  and  LVII  samples.  Note  that  in  the  LVII  sample  only  two  MOS  (63B  and  91A) 
have  at  least  100  soldiers  with  complete  predictor  and  criterion  data.  Also, 
there  are  no  soldiers  with  complete  data  in  the  LVII  sample  for  MOS  31C;  no 
MOS-specific  criteria  or  supervisory  simulation  data  were  administered  to 
soldiers  in  that  MOS,  so  those  soldiers  were  not  included  in  the  LVII 
validation  analyses. 


Table  3.1 


Soldiers  in  LVI  and  LVII  Data  Sets  With  Complete  Predictor  and  Criterion  Data 
by  MOS _ _ _ _ _ _ _ 


MOS 

LVI 

LVII 

IIB 

Infantryman 

235 

83 

13B 

Cannon  Crewmember 

553 

84 

19K 

Ml  Armor  Crewman 

446 

82 

Single  Channel  Radio  Operator 

172 

63B 

Light- Wheel  Ve^hicle  Mechanic 

406 

105 

71L 

Administrative  Specialist 

252 

77 

88M^ 

Motor  Transport  Operator 

221 

37 

91A/B 

Medical  Specialist 

535 

118 

95B 

Mi  1 itary  Police 

270 

93 

Total 

3,163 

679 

*  MOS-specific  and  supervisory  simulation  criterion  data  were  not  collected  for  MOS  31C  in  LVII. 
“  MOS  31C  and  88M  were  not  included  in  LVII  validity  analyses. 


The  number  of  soldiers  in  each  MOS  in  the  LVII  sample  who  vyere  able  to 
meet  the  setwise  deletion  requirements  used  in  the  LVI  analyses  is  shown  in 
Table  3.2.  Even  this  relaxed  sample  selection  strategy  results  in  three  MOS 
(19K,  71L,  and  88M)  with  sample  sizes  of  consistently  fewer  than  100  soldiers 
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Table  3.2 


Soldiers  in  LVII  Sample  Meeting  Setwise  Deletion  Data  Requirements  for 
Validation  of  ASVAB  Operational  Scores  and  Spatial,  Computer,  JOB,  ABLE,  and 
AVOICE  Experimental  Battery  Predictor  Composites  by  MOS _ 


MOS 

Predictor 

Sets 

ASVAB 

Spatial 

Computer 

JOB 

ABLE 

AVOICE 

IIB 

277 

269 

97 

251 

248 

257 

13B 

116 

112 

102 

107 

97 

105 

19K 

103 

89 

90 

84 

88 

88 

63B 

152 

132 

124 

123 

117 

126 

71L 

126 

101 

92 

93 

96 

91 

88M^ 

66 

46 

43 

43 

43 

44 

91A 

155 

143 

133 

139 

128 

137 

95B 

127 

119 

110 

114 

105 

115 

Total 

1,122 

1,011 

791 

954 

922 

963 

^HOS  88M  was  not  included  in  LVII  validity  analyses. 


As  a  result  of  this  finding,  a  third  sample  selection  strategy,  which  we  have 
termed  "predictor/criterion  setwise  deletion,"  was  considered  to  enlarge  the 
samples  for  the  present  analyses.  This  strategy  extends  the  reasoning 
underlying  the  setwise  strategy  employed  in  the  LVI  analyses.  Specifically, 
to  be  included  in  the  validation  sample  for  a  given  criterion/predictor  set 
pair,  soldiers  were  required  to  have  complete  data  for  the  ASVAB,  the 
predictor  composites  in  the  predictor  set  being  examined,  and  only  the 
specific  criterion  score  being  predicted.  Thus,  in  addition  to  not  requiring 
that  a  soldier  have  complete  predictor  data  to  be  included  in  a  particular 
validity  analysis,  the  present  strategy  also  did  not  require  that  the  soldier 
have  complete  data  for  all  of  the  criterion  scores. 

As  expected,  this  strategy  increases  the  sample  sizes  available  for 
estimating  the  validity  of  the  Experimental  Battery  predictors  against  second- 
tour  performance.  Table  3.3  reports  the  number  of  soldiers  in  each  MOS 
meeting  the  predictor/criterion  setwise  deletion  sample  criteria  for  the  Core 
Technical  Proficiency  criterion.  This  table  indicates  that  88M  was  the  only 
MOS  with  fewer  than  100  soldiers  conforming  to  this  selection  strategy, 
regardless  of  the  predictor  set.  Similar  results  and  sample  sizes  were  found 
for  the  other  criterion  constructs  (the  sample  sizes  for  the  full  set  of 
performance  factor  criteria  are  reported  in  Appendix  B). 

Based  on  these  findings,  all  analyses  reported  in  this  chapter  were 
conducted  using  samples  selected  with  the  predictor/criterion  setwise  deletion 
strategy.  However,  because  the  number  of  soldiers  in  88M  was  considerably 
smaller  than  that  of  the  other  MOS,  that  MOS  (along  with  31C)  was  not  included 
in  the  analyses. 
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Table  3.3 


Soldiers  in  LVII  Sample  Meeting  Predictor/Criterion  Setwise  Deletion  Data 
Requirements  for  Validation  of  ASVAB  Operational  Scores  and  Spatial,  Computer, 
JOB,  ABLE,  and  AVOICE  Experimental  Battery  Predictor  Composites  Against  Core 
Technical  Proficiency  by  MOS _ _ _ _ _ 


Predictor  Sets 


MOS 

ASVAB 

Spatial 

Computer 

JOB 

ABLE 

AVOICE 

IIB 

333 

322 

112 

301 

297 

309 

13B 

170 

165 

152 

159 

148 

156 

19K 

156 

130 

130 

122 

123 

129 

63B 

169 

147 

139 

136 

132 

140 

71L 

147 

115 

104 

105 

109 

102 

88M^ 

84 

56 

54 

51 

52 

53 

91A 

205 

191 

174 

185 

165 

183 

95B 

160 

149 

140 

142 

133 

145 

Total 

1,424 

1,275 

1,005 

1,201 

1,159 

1,217 

*M0S  88M  was  not  included  in  LVII  validity  analyses. 


Predictors 

The  predictor  scores  used  in  these  analyses  were  derived  from  the 
operationally  administered  ASVAB  and  the  paper-and-penci  1  and  computerized 
tests  administered  in  the  Project  A  LV  Experimental  Battery.  For  the  ASVAB, 
four  types  of  scores  were  examined.  These  scores,  listed  in  Table  3.4, 
include  the  nine  ASVAB  subtests  (of  which  the  Verbal  score  is  a  composite  of 
the  Word  Knowledge  and  Paragraph  Comprehension  subtests),  the  four  ASVAB 
factor  composite  scores,  the  AFQT,  and  the  MOS-appropriate  Aptitude  Area 
composite  scores. 

The  scores  derived  from  the  LV  Experimental  Battery  are  listed  in  Table 
3.5.  With  one  exception,  these  scores  are  described  by  Peterson  et  al. 

(1990).  The  exception  concerns  the  scores  derived  for  the  ABLE.  Note  that 
three  different  sets  of  ABLE  scores  are  listed  in  Table  3.5.  The  first  set, 
labeled  the  ABLE  Rational  Composites,  were  derived  along  with  the  other  LV 
predictor  composites.  The  other  two  sets,  labeled  ABLE-168  Composites  and 
ABLE-114  Composites,  were  based  on  results  of  factor  analyses  of  the  ABLE 
items.  ABLE-168  was  scored  using  168  of  the  ABLE  items,  and  ABLE-114  was 
scored  using  only  114  items.  The  development  of  these  scores  is  described  by 
White  (1994). 
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Table  3.4 

Four  Sets  of  ASVAB  Scores  Used  in  LVII  Validity  Analyses 


ASVAB  Subtests 

General  Science 
Arithmetic  Reasoning 

Verbal  (Word  Knowledge  and  Paragraph  Comprehension) 

Numerical  Operations 

Coding  Speed 

Auto/Shop  Information 

Mathematical  Knowledge 

Mechanical  Comprehension 

Electronic  Information 


ASVAB  Factor  Composites  .  r  .c  x-  \ 

Technical  (Auto/Shop,  Mechanical  Comprehension,  Electronics  Information) 

Quantitative  (Math  Knowledge,  Arithmetic  Reasoning) 

Verbal  (Word  Knowledge,  Paragraph  Comprehension,  General  Science) 

Speed  (Coding  Speed,  Number  Operations) 


AFQT 

Aptitude  Area  Composites  (1  per  HOS) 
IIB:  CO  (Combat) 

13B:  FA  (Field  Artillery) 

19K:  CO  (Combat) 

63B:  MM  (Mechanical  Maintenance) 
71L:  CL  (Clerical) 

91A:  ST  (Skilled  Technical) 

95B:  ST  (Skilled  Technical) 


Criteria 

The  second-tour  performance  measures  collected  from  the  LVII  sample 
constitute  a  subset  and  extension  of  the  first-tour  measures  collected  from 
the  LVI  sample.  These  measures  include: 

0  Revised  hands-on  job  sample  measures 

0  Revised  job  knowledge  tests 

0  Army-wide  BARS 

0  MOS-specific  BARS 

0  Combat  performance  prediction  ratings 

0  Administrative  indices  of  performance 

0  Supervisory  simulation  exercises 

0  Situational  Judgment  Test 
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Table  3.5 


Battery  Predictor  Scores  Used  in  LVII  Validity  Analyses 


Sets  of  Experimental 

Spatial  Composite 

Spatial 

Computer  Composites 

Psychomotor 
Perceptual  Speed 
Perceptual  Accuracy 
Number  Speed  and  Accuracy 
Basic  Speed 
Basic  Accuracy 
Short-Term  Memory 
Movement  Time 

JOB  Composites 

Autonomy 

High  Expectations 
Routi ne 

AVOICE  Composites 

Administrative 
Audiovisual  Arts 
Food  Service 
Structural /Machines 
Protective  Services 
Rugged/Outdoors 
Social 

Skilled  Technical 


ABLE  Rational  Composites 

Achievement  Orientation 

Adjustment 

Physical  Condition 

Internal  Control 

Cooperativeness 

Dependabi 1 ity 

Leadership 

ABLE-168  Composites 

Locus  of  Control 
Cooperativeness 
Dominance 
Dependabi 1 ity 
Physical  Condition 
Stress  Tolerance 
Work  Orientation 

ABLE-114  Composites 

Locus  of  Control 
Cooperativeness 
Dominance 
Dependabi 1 ity 
Physical  Condition 
Stress  Tolerance 
Work  Orientation 


These  measures  generated  a  set  of  23  basic  scores  that  were  the  basis 
for  the  LVII  performance  modeling  analysis  reported  by  Hanson,  Campbell,  and 
McKee  (in  press).  Those  analyses  examined  the  fit  of  a  preliminary  model  of 
second-tour  performance  that  was  based  on  data  collected  from  the  Project  A 
Concurrent  Validation  second-tour  (CVII)  sample,  as  well  as  the  fit  of  various 
alternative  a  priori  models.  The  model  of  second-tour  performance  emerging 
from  these  analyses  (labeled  the  Leadership  Factor  model)  specified  six 
substantive  performance  factors  and  three  method  factors  ("written  verbal," 
"ratings,"  and  "simulation  exercise").  The  three  methods  factors  were  defined 
to  be  orthogonal  to  the  substantive  factors,  but  the  correlations  among  the 
substantive  factors  were  not  so  constrained. 

The  six  substantive  factors  and  the  variables  that  are  scored  on  each 
are  listed  in  Table  3.6.  Of  the  six  substantive  second-tour  performance 
factors,  three  correspond  exactly  to  performance  factors  in  the  model  of 
first-tour  performance  developed  using  data  collected  from  the  Project  A 
Concurrent  Validation  first-tour  (CVI)  sample  and  confirmed  using  data 
collected  in  LVI  (Oppler,  Childs,  &  Peterson,  1994).  These  factors  are  Core 
Technical  Proficiency  (CTP),  General  Soldiering  Proficiency  (GSP),  and 
Physical  Fitness  and  Military  Bearing  (PFB).  Consistent  with  the  procedures 
used  for  LVI,  the  GSP  factor  scores  created  for  soldiers  in  MOS  IIB  are 
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Table  3.6 

LVII  Performance  Factors  and  the  Basic  Criterion  Scores  That  Define  Them 


Core  Technical  Proficiency  (CTP) 

Hands-On  Test  -  MOS-Specific  Tasks 
Job  Knowledge  Test  -  MOS-Specific  Tasks 

General  Soldiering  Proficiency  (GSP) 

Hands-On  Test  -  Common  Tasks 
Job  Knowledge  Test  -  Common  Tasks 


Achievement  and  Effort  (AE) 

Admin.  Index  -  Number  of  Awards  and  Certificates 
Army-Wide  BARS  Overall  Effectiveness  Rating  Scale 
Army-Wide  BARS  Technical  Skill/Effort  Ratings  Factor 
Average  of  MOS  BARS  Rating  Scales 
Average  of  Combat  Prediction  Rating  Scales 


Personal  Discipline  (PD) 

Admin.  Index  -  Number  of  Articles  15  and  Flag  Actions 
Army-Wide  BARS  Personal  Discipline  Ratings  Factor 


Physical  Fitness  and 
Admin.  Index  - 
Army-Wide  BARS 


Military  Bearing  (PFB) 

Physical  Readiness  Score 

Physical  Fitness/Bearing  Rating  Factor 


Leadership  (LDR) 

Admin.  Index  - 
Army-Wide  BARS 
Supervisory 
Supervisory 
Supervisory 
Supervisory 
Supervisory 
Skills 


Promotion  Grade  Deviation  Score 
Leading/Supervising  Rating  Factor 
Simulation  -  Disciplinary  Structure 
Simulation  -  Disciplinary  Communication 
Simulation  -  Disciplinary  Interpersonal  Skill 
Simulation  -  Counseling  Diagnosis/Prescription 
Simulation  -  Counseling  Communication/Interpersonal 


Supervisory 

Supervisory 

Situational 


Simulation  -  Training 
Simulation  -  Training 
Judgment  Test  -  Total 


Structure 

Motivation 

Score 


treated  as  CTP  scores  in  the  validity  analyses.  (Tasks  that  are  considered 
"general"  to  the  Army  for  soldiers  in  most  other  MOS  are  considered  central  or 
"core"  to  soldiers  in  IIB.) 

Two  of  the  second-tour  performance  factors--Achievement  and  Effort  (AE) 
and  Personal  Discipline  (PD) — have  a  somewhat  different  composition  than  their 
first-tour  counterparts  (Effort  and  Leadership  [ELS]  and  Maintaining  Personal 
Discipline  [MPD],  respectively).  That  is,  the  second-tour  Achievement  and 
Effort  factor  contains  one  score  (the  average  of  the  Combat  Performance 
Prediction  Rating  Scales)  that  was  not  included  in  the  first-tour  Effort  and 
Leadership  factor  and  does  not  include  any  rating  scales  reflective  of 
leadership  performance.  Also,  the  second-tour  Personal  Discipline  factor  is 
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rnissing  one  score  (Promotion  Rete)  that  was  incorporated  in  the  first-tour 
version  of  that  factor.  The  sixth  second-tour  performance  factor--Leadership 
(LDR)--has  no  counterpart  in  the  first-tour  performance  model,  although  it 
does  include  the  Promotion  Rate  score  that  had  previously  been  included  in  the 
first-tour  MPO  factor  and  all  rating  scales  reflective  of  leadership 
performance. 

In  addition  to  the  six  performance  factors,  four  additional  criteria 
were  used  in  the  validation  analyses  reported  in  this  chapter.  Two  of  these 
criteria  are  variations  of  the  Leadership  factor.  The  first  variation  (LDR2) 
does  not  include  the  Situational  Judgment  Test  (SJT)  total  score  that  was 
included  in  the  original  Leadership  factor,  and  the  second  variation  (LDR3) 
does  not  include  either  the  SJT  or  the  scores  from  the  Supervisory  Simulation 
exercises.  The  other  two  criteria  included  in  the  validation  analyses  are  the 
total  scores  from  the  hands-on  and  the  job  knowledge  tests. 

Analyses 

The  analysis  procedure  consisted  of  the  following  steps. 

A.  Using  the  predictor/criterion  setwise  deletion  sample,  multiple 
correlations  between  each  set  of  predictor  scores  and  the 
substantive  factor  scores,  the  two  additional  leadership  factor 
scores,  and  the  total  scores  from  the  hands-on  and  job  knowledge 
tests  were  computed  separately  by  MOS  and  then  averaged. 

1)  As  indicated  above,  ASVAB  was  represented  by 

a)  The  nine  ASVAB  subtest  scores 

b)  The  four  ASVAB  factor  scores 

c)  The  AFQT 

d)  The  MOS-appropriate  Aptitude  Area  composite  score 

2)  ABLE  was  represented  by  three  sets  of  scores: 

a)  The  seven  rational  scales  developed  for  the 
Experimental  Battery 

b)  Seven  empirical  scales  developed  on  the  basis  of  a 
factor  analysis  that  retained  all  of  the  items  (and 
which  used  168  of  them) 

c)  Seven  empirical  scales  developed  by  using  the  results 
of  the  factor  analysis  to  select  the  best  items  to 
reflect  each  factor  (and  which  used  only  114  items) 

Each  of  the  other  predictor  sets  was  represented  by  a  single  set 
of  scores  as  described  above  and  in  Table  3.5. 

Results  were  computed  both  with  and  without  correcting  for 
multivariate  range  restriction  (Lord  &  Novick,  1968).  Corrections 
for  range  restriction  were  made  using  the  9x9  intercorrelation 
matrix  among  the  subtests  in  the  1980  Youth  Population  (Department 
of  Defense,  1982).  All  results  were  adjusted  for  shrinkage  using 
Rozeboom's  (1978)  Formula  8. 
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Using  the  predictor/criterion  setwise  deletion  sample,  incremental 
validities  for  each  set  of  Experimental  Battery  predictors  (e.g., 
AVOICE  composites  or  computer  composites)  over  the  four  ASVAB 
factor  composites  were  computed  against  the  same  criteria  used  to 
compute  the  validities  in  Step  A.  Once  again,  the  results  were 
computed  separately  by  MOS  and  then  averaged.  Also,  the  results 
were  computed  both  with  and  without  correcting  for  range 
restriction,  and  were  adjusted  for  shrinkage  using  the  Rozeboom 
formula. 


RESULTS 

Multiple  Correlations  for  ASVAB  and  LVII  Experimental  Battery 
Predictors  (Based  on  Predictor/Criterion  Deletion  Sample) 

Multiple  Correlations  bv  Predictor  Type 

Multiple  correlations  for  the  four  ASVAB  factors,  the  single  spatial 
composite,  the  eight  computer-based  predictor  scores,  the  three  JOB  composite 
scores,  the  seven  ABLE  composite  scores,  and  the  eight  AVOICE  composite  scores 
are  reported  in  Table  3.7.  Using  the  predictor/criterion  deletion  sample, 
these  results  were  computed  separately  by  MOS  and  then  averaged.  These 
results  have  also  been  adjusted  for  shrinkage  using  the  Rozeboom  formula  and 
corrected  for  range  restriction.  Results  which  have  not  been  corrected  for 
range  restriction  (but  which  have  been  adjusted  for  shrinkage)  are  reported  in 
Table  3.8. 

The  results  in  Table  3.7  indicate  that  the  four  ASVAB  factors  were  the 
best  set  of  predictors  for  all  of  the  criterion  performance  factors  (CTP,  GSP, 
AE,  PD,  PFB,  and  LDR),  the  two  additional  leadership  factors  (LD2  and  LD3), 
and  the  Hands-On  and  Job  Knowledge  total  scores.  The  highest  multiple 
correlation  was  between  the  ASVAB  factors  and  the  Job  Knowledge-Total  score 
(R  =  .74),  while  the  lowest  was  with  the  PD  and  PFB  scores  (R  =  .15  and  .16, 
respectively) . 

With  the  exception  of  the  prediction  of  PFB  with  the  ABLE  composites, 
the  spatial  composite  was  the  next  best  predictor.  The  pattern  of  multiple  Rs 
for  the  spatial  composite  was  highly  similar  to  the  ASVAB  pattern,  with  the 
highest  multiple  R  being  with  the  JK-Total  score  (R  =  .67)  and  the  lowest  with 
the  PD  and  PFB  scores  (R  =  .15  and  .13,  respectively).  Next  in  line  were  the 
eight  computer  composites,  except  for  the  prediction  of  AE,  where  the  ABLE 
composites  had  slightly  higher  validities  (R  =  .09  vs.  R  =  .14),  and  of  PFB, 
where  the  computer  composite  had  the  smallest  R. 

The  three  JOB  composites,  the  seven  ABLE  composites,  and  the  eight 
AVOICE  composites  had  different  patterns  of  multiple  Rs  for  the  different 
criterion  performance  factors.  The  AVOICE  was  highest  among  the  three  for 
CTP,  GSP,  LDR,  and  the  JK-Total  score,  while  JOB  was  highest  for  the  HO-Total 
criterion;  ABLE  was  highest  for  AE ,  PFB,  LDR2,  and  LDR3. 
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Table  3.7 


Mean  of  Multiple  Correlations  Computed  Within  Job  for  LVII  Samples  for  ASVAB 
Factors,  Spatial,  Computer,  JOB,  ABLE  Composites,  and  AVOICE:  Corrected  for 


Range  Restriction 


.  .  .  . 

Criterion^ 

No.  of 
MOS*’ 

ASVAB 

Factors 

[4] 

Spatial 

[1] 

Computer 

[8] 

JOB 

[3] 

ABLE  Comp. 
[7] 

AVOICE 

[8] 

CTP 

7 

64 

(10) 

57 

(11) 

53 

(11) 

33  (17) 

24 

(19) 

41  (12) 

GSP 

6 

63 

(06) 

58 

(05) 

48 

(10) 

28  (19) 

19 

(17) 

29  (24) 

AE 

7 

29 

(14) 

27 

(13) 

09 

(11) 

07  (12) 

13 

(17) 

09  (15) 

PD 

7 

15 

(12) 

15 

(10) 

12 

(12) 

03  (05) 

06 

(10) 

06  (10) 

PFB 

7 

16 

(11) 

13 

(06) 

03 

(06) 

07  (08) 

17 

(15) 

09  (13) 

LDR 

7 

63 

(14) 

55 

(08) 

49 

(13) 

34  (21) 

34 

(20) 

35  (24) 

LDR2 

7 

51 

(16) 

46 

(12) 

35 

(19) 

26  (21) 

25 

(22) 

25  (24) 

LDR3 

7 

47 

(13) 

39 

(12) 

31 

(15) 

19  (18) 

23 

(17) 

20  (21) 

HO-Total 

7 

46 

(13) 

41 

(14) 

33 

(21) 

24  (11) 

12 

(15) 

21  (18) 

JK-Total 

7 

74 

(05) 

67 

(03) 

58 

(06) 

37  (16) 

29 

(17) 

44  (14) 

Note;  Predictor/criterion  setwise  deletion  sample.  Adjusted  for  shrinkage 

(Rozeboom  formula  8).  Numbers  in  brackets  are  the  numbers  of  predictor 
scores  entering  prediction  equations.  Numbers  in  parentheses  are 
standard  deviations.  Decimals  omitted. 

^  CTP  =  Core  Technical  Proficiency;  GSP  =  General  Soldiering  Proficiency; 

AE  =  Achievement  and  Effort;  PD  =  Personal  Discipline; 

PFB  =  Physical  Fitness/Military  Bearing;  LDR  =  Leadership; 

LDR2  =  Leadership  minus  Situational  Judgment  Test; 

LDR3  =  Leadership  minus  Situational  Judgment  Test  and  Supervisory 
Simulation  Exercises;  HO  =  Hands-On;  JK  =  Job  Knowledge 
^  Number  of  MOS  for  which  validities  were  computed. 
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Table  3.8 


Mean  of  Multiple  Correlations  Computed  Within  Job  for  LVII  Samples  for  ASVAB 
Factors,  Spatial,  Computer,  JOB,  ABLE  Composites,  and  AVOICE:  Uncorrected  for 
Range  Restriction _ _ _ _ _ 


ASVAB 

No.  of  Factors  Spatial  Computer  JOB  ABLE  Comp.  AVOICE 
Criterion  MOS^ _ [4] _ [^j _  [^3 _ [^3 _ _ [®J _ 


CTP  7 

GSP  6 

AE  7 

PD  7 

PFB  7 

LDR  7 

L0R2  7 

LDR3  7 

HO-Total  7 

JK-Total  7 


40 

(12) 

34 

(12) 

48 

(06) 

42 

(04) 

12 

(09) 

14 

(03) 

02 

(03) 

09 

(06) 

09 

(12) 

08 

(06) 

38 

(12) 

33 

(05) 

30 

(11) 

26 

(04) 

26 

(07) 

19 

(05) 

28 

(13) 

25 

(12) 

56 

(08) 

47 

(06) 

20 

(15) 

11 

(13) 

21 

(15) 

13 

(15) 

00 

(00) 

00 

(00) 

03 

(06) 

02 

(04) 

00 

(00) 

05 

(08) 

09 

(14) 

12 

(15) 

08 

(14) 

08 

(12) 

06 

(12) 

03 

(07) 

12 

(17) 

06 

(07) 

24 

(13) 

18 

(15) 

00 

(00) 

13 

(15) 

02 

(05) 

15 

(13) 

07 

(10) 

02 

(06) 

05 

(10) 

03 

(06) 

15 

(15) 

06 

(11) 

11 

(14) 

12 

(15) 

09 

(12) 

08 

(11) 

04 

(07) 

06 

(10) 

00 

(00) 

08 

(12) 

02 

(05) 

19 

(16) 

Note;  Predictor/criterion  setwise  deletion  sample.  Adjusted  for  shrinkage 
(Rozeboom  formula  8).  Numbers  in  brackets  are  the  numbers  of 
predictor  scores  entering  prediction  equations.  Numbers  in 
parentheses  are  standard  deviations.  Decimals  omitted. 

®  Number  of  MOS  for  which  validities  were  computed. 
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In  general,  with  regard  to  the  ABLE,  the  highest  correlations  are  with 
the  Leadership  factor  and  the  Core  Technical  Performance  factor.  Compar¬ 
atively,  the  correlations  of  the  ABLE  with  Achievement  and  Effort  and  with 
Personal  Discipline  are  lower.  In  large  part  this  reflects  the  emergence  of  a 
separate  leadership  factor  and  the  fact  that  the  promotion  rate  index  produced 
a  better  fit  for  the  LVII  model  when  it  was  scored  as  a  Leadership  component 
than  as  a  component  of  the  Personal  Discipline  factor.  As  in  CVII,  a  faster 
promotion  rate  for  LVII  personnel  is  more  a  function  of  good  things  that 
happen  rather  than  an  absence  of  negative  events  that  act  to  slow  an 
individual's  progression,  as  it  was  in  CVI  and  LVI. 

When  these  results  are  not  corrected  for  range  restriction  (Table  3.8), 
all  of  the  multiple  Rs  were  lowered,  some  quite  dramatically.  Overall,  the 
ASVAB  factors  still  had  the  highest  multiple  Rs,  with  the  spatial  composite 
again  having  a  highly  similar  (but  slightly  smaller  in  magnitude)  pattern. 

(It  should  be  noted  that  differences  between  the  patterns  of  the  corrected  and 
uncorrected  results  could  arise  due  to  differences  in  range  restriction  across 
the  various  predictors.  Therefore,  where  differences  between  the  two  sets  of 
results  are  reported,  we  believe  that  the  corrected  results  should  be  given 
greater  emphasis,  since  differences  between  measures  with  respect  to  range 
restriction  will  have  been  removed.) 

Comparisons  of  Alternative  ASVAB  Scores 

The  average  multiple  correlations  (corrected  and  uncorrected  for  range 
restrictions,  respectively)  for  the  four  different  sets  of  ASVAB  scores  are 
reported  in  Tables  3.9  and  3.10.  The  corrected  results  indicate  that  all  four 
sets  had  virtually  identical  multiple  Rs  with  all  of  the  criterion  measures. 
The  four  ASVAB  factors  tended  to  have  very  slightly  higher  validities  than  the 
other  three  sets  of  scores,  whereas  the  AFQT  tended  to  have  slightly  lower 
validities. 

The  uncorrected  correlations  again  showed  highly  consistent  multiple  Rs 
for  the  four  different  sets  of  ASVAB  scores  (Table  3.10);  as  was  true 
previously,  the  uncorrected  Rs  were  much  smaller  than  the  corrected 
correlations.  For  a  few  criteria  --  AE,  PFB,  and  LDR2  and  LDR3  --  the  multiple 
Rs  for  ASVAB  subtests  were  smaller  than  the  other  three  sets. 

Comparisons  of  Alternative  ABLE  Scores 

The  average  multiple  correlations  for  the  three  sets  of  ABLE  scores  are 
reported  in  Tables  3.11  and  3.12  (corrected  and  uncorrected,  respectively). 

The  results  in  Table  3.11  indicate  that  the  pattern  and  levels  of  multiple  Rs 
were  generally  very  similar  across  the  three  sets.  However,  the  ABLE 
composites  were  somewhat  better  predictors  of  LDR  (R  =  .34)  than  were  the 
ABLE-168  and  ABLE-114  factor  scores  (R  =  .26  and  .27,  respectively),  and  the 
ABLE-168  factor  scores  were  the  best  predictors  of  CTP  (R  =  .30  versus  R  =  .24 
for  the  ABLE  composites  and  R  =  .27  for  the  ABLE-114  factor  scores).  Note 
that  the  uncorrected  results  reported  in  Table  3.11  are  much  smaller  than  the 
results  in  Table  3.10,  but  the  pattern  and  levels  of  the  validities  are  still 
very  similar  across  the  three  sets  of  scores. 


134 


Table  3.9 


Mean  of  Multiple  Correlations  Computed  Within  Job  for  LVII  Samples  for  ASVAB 
Subtests,  ASVAB  Factors,  AFQT,  and  ASVAB  MOS-Appropr iate  Aptitude  Area: 


Criterion 

No.  of 
MOS^ 

ASVAB 

Subtests 

[9] 

ASVAB 

Factors 

[4] 

AFQT 

[1] 

Aptitude 

Area° 

[1] 

CTP 

7 

64  (10) 

64  (10) 

61  (08) 

63  (09) 

GSP 

6 

62  (07) 

63  (06) 

58  (09) 

61  (07) 

AE 

7 

28  (10) 

29  (14) 

28  (13) 

30  (11) 

PD 

7 

12  (11) 

15  (12) 

16  (11) 

16  (10) 

PFB 

7 

10  (08) 

16  (11) 

16  (06) 

14  (07) 

LDR 

7 

64  (10) 

63  (14) 

62  (15) 

59  (14) 

LDR2 

7 

51  (13) 

51  (16) 

50  (16) 

49  (16) 

LDR3 

7 

46  (13) 

47  (13) 

45  (14) 

44  (14) 

HO-Total 

7 

45  (12) 

46  (13) 

43  (13) 

46  (11) 

JK-Total 

7 

74  (05) 

74  (05) 

70  (06) 

72  (06) 

Note:  Predictor/criterion  setwise  deletion  samples.  Adjusted  for  shrinkage 

(Rozeboom  formula  8).  Numbers  in  brackets  are  the  numbers  of  predictor 
scores  entering  prediction  equations.  Numbers  in  parentheses  are 
standard  deviations.  Decimals  omitted. 

^  Number  of  MOS  for  which  validities  were  computed. 

^  MOS-Appropriate  Aptitude  Area  composite. 
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Table  3.10 


Mean  of  Multiple  Correlations  Computed  Within  Job  for  LVII  Samples  for  ASVAB 
Subtests,  ASVAB  Factors,  AFQT,  and  ASVAB  MOS-Appropr iate  Aptitude  Area: 
Uncorrected  for  Range  Restriction _ _ _ 


Criterion 

No.  of 
MOS^ 

ASVAB 

Subtests 

[9] 

ASVAB 

Factors 

[4] 

AFQT 

[1] 

Aptitude 

Area“ 

[1] 

CTP 

7 

40  (10) 

40  (12) 

39  (06) 

42  (09) 

GSP 

6 

45  (08) 

48  (06) 

43  (08) 

44  (06) 

AE 

7 

03  (07) 

12  (09) 

14  (08) 

16  (08) 

PD 

7 

01  (02) 

02  (03) 

11  (06) 

09  (06) 

PFB 

7 

02  (05) 

09  (12) 

10  (09) 

09  (08) 

LDR 

7 

36  (17) 

38  (12) 

40  (10) 

35  (09) 

LDR2 

7 

25  (14) 

30  (11) 

30  (10) 

28  (10) 

LDR3 

7 

19  (13) 

26  (07) 

24  (11) 

24  (11) 

HO-Total 

7 

24  (14) 

28  (13) 

27  (08) 

31  (08) 

JK-Total 

7 

55  (09) 

56  (08) 

51  (05) 

52  (06) 

Note:  Predictor/criterion  setwise  deletion  samples.  Adjusted  for  shrinkage 

(Rozeboom  formula  8).  Numbers  in  brackets  are  the  numbers  of  predictor 
scores  entering  prediction  eguations.  Numbers  in  parentheses  are 
standard  deviations.  Decimals  omitted. 

®  Number  of  MOS  for  which  validities  were  computed. 

MOS-Appropriate  Aptitude  Area  composite. 
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Table  3.11 


Mean  of  Multiple 
Composites,  ABLE- 

Correlations  Computed  Within  Job  for  LVII  Samples  for  ABLE 
-168,  and  ABLE-114:  Corrected  for  Range  Restriction 

Criterion 

No.  of 
MOS^ 

ABLE 

Composites 

[7] 

ABLE- 

168 

[7] 

ABLE- 

114 

[7] 

CTP 

7 

24  (19) 

30  (10) 

27 

(15) 

GSP 

6 

19  (17) 

18  (16) 

18 

(17) 

AE 

7 

13  (17) 

12  (14) 

11 

(17) 

PD 

7 

06  (10) 

05  (09) 

04 

(07) 

PFB 

7 

17  (15) 

16  (16) 

16 

(14) 

LDR 

7 

34  (20) 

26  (25) 

27 

(25) 

LDR2 

7 

25  (22) 

24  (23) 

25 

(23) 

LDR3 

7 

23  (17) 

19  (18) 

20 

(19) 

HO-Total 

7 

12  (15) 

12  (13) 

14 

(14) 

JK-Total 

7 

29  (17) 

30  (13) 

28 

(15) 

Note:  Predictor/criterion  setwise  deletion  samples.  Adjusted  for  shrinkage 

(Rozeboom  formula  8).  Numbers  in  brackets  are  the  numbers  of 
predictor  scores  entering  prediction  equations.  Numbers  in 
parentheses  are  standard  deviations.  Decimals  omitted. 

^  Number  of  MOS  for  which  validities  were  computed. 


Table  3.12 


Mean  of  Multiple  Correlations  Computed  Within  Job  for  LVII  Samples  for  ABLE 
Composites,  ABLE-168,  and  ABLE-114;  Uncorrected  for  Range  Restriction _ 


Criterion 

No.  of 
MOS® 

ABLE 

Composites 

[7] 

ABLE- 

168 

[7] 

ABLE- 

114 

[7] 

CTP 

7 

00  (00) 

00  (00) 

00  (00) 

GSP 

6 

02  (05) 

03  (06) 

00  (00) 

AE 

7 

07  (10) 

03  (05) 

04  (08) 

PD 

7 

05  (10) 

05  (10) 

04  (10) 

PFB 

7 

15  (15) 

15  (14) 

16  (13) 

LDR 

7 

11  (14) 

09  (10) 

08  (11) 

LDR2 

7 

09  (12) 

08  (10) 

07  (08) 

LDR3 

7 

04  (07) 

00  (00) 

00  (00) 

HO-Total 

7 

00  (00) 

01  (03) 

01  (04) 

JK-Total 

7 

02  (05) 

03  (07) 

00  (00) 

Note:  Predictor/criterion  setwise  deletion  samples.  Adjusted  for  shrinkage 

(Rozeboom  formula  8).  Numbers  in  brackets  are  the  numbers  of 
predictor  scores  entering  prediction  equations.  Numbers  in 
parentheses  are  standard  deviations.  Decimals  omitted. 

®  Number  of  MOS  for  which  validities  were  computed. 


e 
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Incremental  Validities  for  the  Experimental  Battery  Predictors 
Over  the  ASVAB  Factors  (Based  on  Predictor/Criterion  Deletion  Sample) 


Incremental  validity  results  for  the  Experimental  Battery  predictors 
over  the  ASVAB  factor  composites  are  reported  in  Tables 
(Results  corrected  for  range  restriction  are  reported  in  Tables  d.id  ana 
uncorrected  results  in  Tables  3.15  and  3.16.)  Table  3.13  reports  the  multiple 
correlations  for  the  four  ASVAB  factor  composites  alone  (as  computed 
separately  in  each  of  the  predictor/criterion  deletion  samples),  whereas  Table 
3.14  reports  the  multiple  correlations  for  the  four  ASVAB  factors  along  with 
each  set  of  predictors  in  the  Experimental  Battery.  Numbers  underlined  in 
Table  3.14  indicate  multiple  correlations  that  are  higher  than  those  based  on 

ASVAB  alone. 


The  results  indicate  that  there  were  no  increments  to  the  prediction  of 
any  of  the  criteria  for  the  computer,  JOB,  or  AVOICE  composites.  The  spatial 
composite  added  a  point  to  the  prediction  of  GSP ,  AE,  and  JK-Total,  and  the 
ABLE  composites  added  five  points  to  the  prediction  of  PFB  (from  R  -  .13  tor 
ASVAB  alone  to  R  =  .18  for  ASVAB  and  ABLE  together)  and  one  point  to  the 
prediction  of  LDR  (from  R  =  .64  to  R  =  .65). 

The  incremental  validity  results  that  are  not  corrected  for  range 
restriction  (Tables  3.15  and  3.16)  are  similar  in  pattern  to  the  corrected 
results.  Again,  the  largest  increase  in  validity  was  for  the  PFB  criterion 
factor  when  the  ABLE  composites  were  added  to  the  ASVAB  factor  composites  --  a 
change  from  R  =  .10  to  R  =  .17. 

The  adjusted  incremental  validity  results  for  each  of  the  three  sets  of 
ABLE  scores  over  the  ASVAB  factors  are  reported  in  Tables  3.17  and  3.18 
(corrected  and  uncorrected  for  range  restriction,  respectively).  The 
corrected  results  in  Table  3.17  indicate  that  all  three  sets  of  scores  account 
for  variance  in  PFB  not  predicted  by  the  ASVAB  factors,  and  the  ABLE 
composites  and  ABLE-168  factor  scores  add  to  the  prediction  of  LDR.  In 
comparison,  the  uncorrected  results  indicate  that  all  three  sets  of  scores 
provided  incremental  validity  for  both  PD  and  PFB,  but  only  the  ABLE 
composites  add  to  the  prediction  of  LDR. 
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Table  3.13 


Mean  of  Multiple  Correlations  Computed  Within  Job  for  ASVAB  Factors  Within 
Each  LVII  Predictor/Criterion  Setwise  Deletion  Sample:  Corrected  for  Range 


Restriction 


ASVAB 

ASVAB 

ASVAB 

ASVAB 

ASVAB 

Factors 

Factors 

Factors 

Factors 

Factors 

No.  of  (Spatial 

(Computer 

(JOB 

(ABLE  Comp. 

(AVOICE 

Sample) 

Sample) 

Sample) 

Sample) 

Sample) 

Criterion  MOS^  [4] 

[4] 

[4] 

[4] 

[4] 

CTP 

7 

65 

(10) 

64 

(07) 

66 

(11) 

67 

(09) 

66 

(12) 

GSP 

6 

61 

(08) 

62 

(09) 

60 

(09) 

61 

(11) 

61 

(08) 

AE 

7 

31 

(15) 

24 

(13) 

32 

(15) 

30 

(17) 

30 

(16) 

PD 

7 

17 

(12) 

21 

(17) 

17 

(13) 

15 

(11) 

16 

(12) 

PFB 

7 

15 

(10) 

17 

(14) 

16 

(10) 

13 

(13) 

18 

(11) 

LDR 

7 

63 

(13) 

62 

(16) 

63 

(14) 

64 

(12) 

62 

(13) 

LDR2 

7 

52 

(17) 

50 

(20) 

50 

(19) 

51 

(20) 

49 

(19) 

LDR3 

7 

46 

(19) 

45 

(23) 

44 

(22) 

45 

(23) 

43 

(23) 

HO-Total 

7 

46 

(14) 

45 

(17) 

44 

(17) 

46 

(18) 

47 

(16) 

JK-Total 

7 

74 

(05) 

74 

(06) 

74 

(06) 

75 

(05) 

74 

(06) 

Note:  Adjusted  for  shrinkage  (Rozeboom  formula  8).  Numbers  in  brackets  are 
the  numbers  of  predictor  scores  entering  prediction  equations. 

Numbers  in  parentheses  are  standard  deviations.  Decimals  omitted. 

^  Number  of  MOS  for  which  validities  were  computed. 
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Table  3.14 

Mean  of  Incremental  Correlations  Over  ASVAB  Factors  Computed  Within  Job  for 
LVII  Samples  for  Spatial,  Computer,  JOB,  ABLE  Composites,  and  AVOICE: 
Corrected  for  Range  Restriction _ _ _ _ _ 


ASVAB 

Factors  (A4) 

A4  + 

A4  + 

A4  +  ABLE 

A4  + 

No.  of  +  Spatial 

Computer 

JOB 

Composites 

AVOICE 

Criterion  MOS^  [5] 

[12] 

[7] 

[11] 

[12] 

CTP 

7 

65 

(11) 

63 

(10) 

65 

(11) 

65 

(11) 

64 

(13) 

GSP 

6 

62 

(08) 

62 

(10) 

60 

(11) 

60 

(10) 

57 

(12) 

AE 

7 

33 

(10) 

10 

(14) 

30 

(15) 

24 

(20) 

20 

(18) 

PD 

7 

16 

(12) 

16 

(17) 

14 

(15) 

13 

(12) 

06 

(11) 

PFB 

7 

12 

(10) 

08 

(11) 

13 

(11) 

18 

(15) 

11 

(l6) 

LDR 

7 

63 

(12) 

61 

(15) 

63 

(13) 

(13) 

62 

(13) 

LDR2 

7 

51 

(17) 

48 

(20) 

49 

(20) 

50 

(24) 

48 

(19) 

LDR3 

7 

45 

(20) 

41 

(21) 

42 

(23) 

45 

(22) 

40 

(24) 

HO-Total 

7 

46 

(15) 

43 

(17) 

44 

(15) 

43 

(21) 

39 

(23) 

JK-Total 

7 

25 

(05) 

73 

(06) 

74 

(06) 

74 

(06) 

73 

(06) 

Note*  Predictor/criterion  setwise  deletion  samples.  Adjusted  for  shrinkage 

(Rozeboom  formula  8).  Numbers  in  brackets  are  the  numbers  of  predictor 
scores  entering  prediction  equations.  Numbers  in  parentheses  are 
standard  deviations.  Underlined  numbers  denote  multiple  Rs  greater 
than  for  ASVAB  Factors  alone  (as  reported  in  Table  3.13).  Decimals 
omitted. 

®  Number  of  MOS  for  which  validities  were  computed. 
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Table  3.15 


Mean  of  Multiple  Correlations  Computed  Within  Job  for  ASVAB  Factors  Within 
LVII  Predictor/Criterion  Setwise  Deletion  Sample:  Uncorrected  for  Range 


Restriction 


ASVAB 

ASVAB 

ASVAB 

ASVAB 

ASVAB 

Factors 

Factors 

Factors 

Factors 

Factors 

No.  of  (Spatial) 

(Computer) 

(JOB) 

(ABLE  Comp.) 

(AVOICE) 

Criterion  MOS^  [4] 

[4] 

[4] 

[4] 

[4] 

CTP 

7 

41 

(11) 

41 

(10) 

42 

(11) 

42 

(09) 

42 

(11) 

GSP 

6 

45 

(09) 

46 

(09) 

44 

(11) 

44 

(11) 

44 

(12) 

AE 

7 

13 

(10) 

07 

(07) 

16 

(08) 

12 

(12) 

12 

(08) 

PD 

7 

04 

(06) 

05 

(06) 

04 

(07) 

02 

(04) 

03 

(04) 

PFB 

7 

08 

(11) 

09 

(13) 

09 

(12) 

10 

(13) 

08 

(11) 

LDR 

7 

39 

(09) 

40 

(10) 

38 

(08) 

38 

(11) 

37 

(10) 

LDR2 

7 

29 

(12) 

29 

(14) 

28 

(15) 

27 

(15) 

27 

(15) 

LDR3 

7 

25 

(12) 

26 

(12) 

25 

(13) 

25 

(12) 

25 

(12) 

HO-Total 

7 

26 

(16) 

28 

(13) 

24 

(18) 

25 

(15) 

27 

(15) 

JK-Total 

7 

56 

(08) 

55 

(10) 

55 

(07) 

56 

(08) 

54 

(09) 

Note:  Adjusted  for  shrinkage  (Rozeboom  formula  8).  Numbers  in  brackets  are 
the  numbers  of  predictor  scores  entering  prediction  equations.  Numbers 
in  parentheses  are  standard  deviations.  Decimals  omitted. 

®  Number  of  MOS  for  which  validities  were  computed. 
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Table  3.16 


Mean  of  Incremental  Correlations  Over  ASVAB  Factors  Computed  Within  Job  for 
LVII  Samples  for  Spatial,  Computer,  JOB,  ABLE  Composites,  and  AVOICE: 
Uncorrected  for  Range  Restriction _ _ _ 


Criterion 

No.  of 
MOS^ 

ASVAB 

Factors  (A4) 

+  Spatial 
[5] 

A4  + 

Computer 

[12] 

A4  + 

JOB 

[7] 

A4  +  ABLE 
Composites 
[11] 

A4  + 
AVOICE 
[12] 

CTP 

7 

41  (13) 

37  (15) 

40  (12) 

35  (14) 

36  (16) 

GSP 

6 

47  (09) 

45  (10) 

42  (16) 

42  (10) 

38  (19) 

AE 

7 

14  (07) 

00  (00) 

09  (09) 

10  (11) 

03  (09) 

PD 

7 

03  (05) 

03  (08) 

06  (10) 

05  (09) 

01  (03) 

PFB 

7 

07  (10) 

03  (05) 

08  (11) 

17  (13) 

08  (13) 

LDR 

7 

39  (09) 

31  (18) 

37  (11) 

39  (13) 

34  (17) 

LDR2 

7 

28  (12) 

20  (20) 

27  (15) 

26  (17) 

22  (17) 

LDR3 

7 

23  (12) 

14  (13) 

20  (15) 

19  (15) 

17  (14) 

HO-Total 

7 

26  (17) 

22  (20) 

22  (17) 

21  (16) 

18  (19) 

JK-Total 

7 

^  (08) 

53  (10) 

55  (09) 

53  (10) 

50  (14) 

Note:  Predictor/criterion  setwise  deletion  samples.  Adjusted  for  shrinkage 

(Rozeboom  formula  8).  Numbers  in  brackets  are  the  numbers  of  predictor 
scores  entering  prediction  eguations.  Numbers  in  parentheses  are 
standard  deviations.  Underlined  numbers  denote  multiple  Rs  greater 
than  for  ASVAB  Factors  alone  (as  reported  in  Table  3.15).  Decimals 
omitted. 

^  Number  of  MOS  for  which  validities  were  computed. 
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Table  3.17 


Mean  of  Incremental  Correlations  Over  ASVAB  Factors  Computed  Within  Job  for 
LVII  Samples  for  ABLE  Composites,  ABLE-168,  and  ABLE-114:  Corrected  for  Range 


Restriction 


ASVAB 

A4  + 

A4  + 

A4  + 

Factors 

ABLE 

ABLE- 

ABLE- 

No.  of 

{A4) 

Comp. 

168 

114 

Criterion 

MOS® 

[4] 

[11] 

[11] 

[11] 

CTP 

7 

67 

(09) 

GSP 

6 

61 

(11) 

AE 

7 

30 

(17) 

PD 

7 

15 

(11) 

PFB 

7 

13 

(13) 

LDR 

7 

64 

(12) 

LDR2 

7 

51 

(20) 

LDR3 

7 

45 

(23) 

HO-Total 

7 

46 

(18) 

JK-Total 

7 

75 

(05) 

65 

(11) 

65 

(11) 

65 

(11) 

60 

(10) 

59 

(13) 

59 

(13) 

24 

(20) 

19 

(22) 

21 

(21) 

13 

(12) 

14 

(13) 

13 

(11) 

18 

(15) 

16 

(16) 

17 

(17) 

65 

(13) 

65 

(13) 

64 

(13) 

50 

(24) 

50 

(25) 

49 

(25) 

45 

(22) 

45 

(22) 

44 

(22) 

43 

(21) 

42 

(21) 

42 

(21) 

74 

(06) 

74 

(06) 

74 

(06) 

Note:  Predictor/criterion  setwise  deletion  samples.  Adjusted  for  shrinkage 

(Rozeboom  formula  8).  Numbers  in  brackets  are  the  numbers  of  predictor 
scores  entering  prediction  equations.  Numbers  in  parentheses  are 
standard  deviations.  Multiple  Rs  for  ASVAB  Factors  alone  are  in 
italics.  Underlined  numbers  denote  multiple  Rs  greater  than  for  ASVAB 
Factors  alone.  Decimals  omitted. 

^  Number  of  MOS  for  which  validities  were  computed. 
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Table  3.18 


Mean  of  Incremental  Correlations  Over  ASVAB  Factors  Computed  Within  Job  for 
LVII  Samples  for  ABLE  Composites,  ABLE-168,  and  ABLE-114:  Uncorrected  for 
Range  Restriction _ _ _ _ _ _ _ 


Criterion 

No.  of 
MOS" 

ASVAB 

Factors 

(A4) 

[4] 

A4  + 

ABLE 

Comp. 

[11] 

A4  + 

ABLE- 

168 

[11] 

A4  + 

ABLE- 

114 

[11] 

CTP 

7 

42  (09) 

35  (14) 

36  (13) 

35  (12) 

GSP 

6 

44  (11) 

42  (10) 

39  (20) 

38  (20) 

AE 

7 

12  (12) 

10  (11) 

05  (08) 

08  (10) 

PD 

7 

02  (04) 

05  (09) 

06  (11) 

W  (09) 

PFB 

7 

10  (13) 

17  (13) 

15  (14) 

16  (15) 

LDR 

7 

38  (11) 

39  (13) 

38  (13) 

M  (12) 

LDR2 

7 

^7  (15) 

26  (17) 

27  (16) 

26  (16) 

LDR3 

7 

25  (12) 

19  (15) 

20  (13) 

17  (13) 

HO-Total 

7 

25  (15) 

21  (16) 

17  (18) 

18  (18) 

JK-Total 

7 

56  (08) 

53  (10) 

53  (11) 

53  (10) 

Note:  Predictor/criterion  setwise  deletion  samples.  Adjusted  for  shrinkage 

(Rozeboom  formula  8).  Numbers  in  brackets  are  the  numbers  of  predictor 
scores  entering  prediction  eguations.  Numbers  in  parentheses  are 
standard  deviations.  Multiple  Rs  for  ASVAB  Factors  alone  are  in 
italics.  Underlined  numbers  denote  multiple  Rs  greater  than  for  ASVAB 
Factors  alone.  Decimals  omitted. 

®  Number  of  MOS  for  which  validities  were  computed. 
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COMPARISON  BETWEEN  VALIDITY  RESULTS  OBTAINED 
WITH  LVI  AND  LVII  SAMPLES 

The  final  set  of  results  concern  the  comparison  between  the  validity 
results  computed  for  the  first-tour  Longitudinal  Validation  (LVI  sample  and 
those  reported  for  the  second-tour  Longitudinal  Validation  (LVII)  sample. 

Levels  of  Validity 

The  multiple  correlations  for  the  ASVAB  factors  and  each  set  of 
experimental  predictors  as  computed  for  LVI  and  LVII,  respectively,  are 
reported  in  Table  3.19.  These  results  have  been  corrected  for  range 
restriction  and  adjusted  for  shrinkage. 

Note  that  the  first-tour  results  are  based  on  the  LVI  samples  selected 
using  the  setwise  deletion  strategy  described  above  for  LVI,  in  comparison  to 
the  predictor/criterion  setwise  deletion  strategy  used  for  LVII.  Thus,  the 
soldiers  included  in  the  LVI  validation  samples  were  required  to  have  complete 
criterion  data,  but  the  soldiers  in  the  LVII  sample  were  not.  Also,  the  LVII 
analyses  did  not  include  two  MOS  (31C  and  88M)  that  were  included  in  the  LVI 
analyses.  Finally,  as  described  earlier,  there  were  differences  between  the 
components  of  the  Achievement  and  Effort  (AE)  and  Personal  Discipline  (PD) 
factors  computed  for  soldiers  in  the  LVII  sample  and  their  corresponding 
factors  in  the  LVI  sample  (Effort  and  Leadership  [ELS]  and  Maintaining 
Personal  Discipline  [MPD],  respectively). 

The  results  in  Table  3.19  demonstrate  that  the  patterns  and  levels  of 
validities  are  very  similar  across  the  two  sets  of  analyses,  especially  for 
the  four  ASVAB  factor  composites.  For  example,  the  multiple  R  between  the 
ASVAB  and  the  CTP  is  .63  and  .64  for  the  LVI  and  LVII  samples,  respectively. 

The  greatest  discrepancies  between  the  two  sets  of  results  concern  the 
multiple  correlations  between  the  ABLE  composites  and  two  of  the  three  will 
do"  criterion  factors--[Maintaining]  Personal  Discipline  and  Physical  Fitness 
and  Military  Bearing.  Specifically,  the  multiple  R  between  ABLE  and  the 
discipline  factor  decreases  from  .15  in  LVI  to  .06  in  LVII,  and  the  multiple  R 
between  ABLE  and  Physical  Fitness  and  Military  Bearing  decreases  from  .28  to 
.17. 


Some  of  the  decrease  in  the  ABLE's  ability  to  predict  Personal 
Discipline  in  LVII  may  be  due  to  the  removal  from  that  factor  of  the  Promotion 
Rate  component.  The  validities  of  the  other  predictors  were  not  similarly 
affected  by  this  scoring  change.  Again,  the  highest  correlation  for  the  ABLE 
in  LVII  was  with  the  Leadership  factor  (R  =  .34).  The  LVII  Leadership  factor 
included  the  promotion  rate  index,  all  scores  derived  from  the  supervisory 
role  plays,  and  the  Army-wide  BARS  (rating  scale)  leading  and  supervising 
factor,  which  was  part  of  the  ELS  factor  in  CVI  and  LVI.  In  effect,  these 
(j i f f 0Y'0nces  were  expected  to  decrease  the  ABLE  correlations  with  the  LVII 
Achievement  and  Effort  factor  and  increase  the  ASVAB  correlations  with  this 
factor  which  in  LVII  is  more  reflective  of  technical  achievement  than  was  the 
ELS  fakor  in  LVI  and  CVI.  These  are  the  expected  patterns  and  they  lend 
further  support  to  the  construct  validity  of  the  performance  models. 


146 


Table  3.19 


Comparison  of  Mean  Multiple  Correlations  Computed  Within  Job  for  ASVAB 
Factors,  Spatial,  Computer,  JOB,  ABLE  Composites,  and  AVOICE  Within  LVI  and 
LVII  Samples:  Corrected  for  Range  Restriction  _ _ _ 


ASVAB 

ABLE 

No 

.  of 

Factors 

Spatial 

Computer 

JOB 

Comp. 

AVOICE 

MOS^ 

[4] 

[1] 

[8] 

[3] 

[7] 

[8] 

Criterion^ 

LVI 

LVII 

LVI  1 

LVII 

LVI 

LVII 

LVI 

LVII 

LVI 

LVII 

LVI 

LVII 

LVI 

LVII 

CTP 

9 

7 

63 

64 

58 

57 

49 

53 

31 

33 

21 

24 

39 

41 

GSP 

8 

6 

66 

63 

65 

58 

55 

48 

32 

28 

24 

19 

38 

29 

ELS/AE 

9 

7 

34 

29 

33 

27 

30 

09 

19 

07 

12 

13 

20 

09 

MPD/PD 

9 

7 

16 

15 

14 

15 

10 

12 

06 

03 

15 

06 

05 

06 

PFB 

9 

7 

12 

16 

08 

13 

13 

03 

07 

07 

28 

17 

09 

09 

HO-Total 

9 

7 

50 

46 

50 

41 

38 

33 

20 

24 

13 

12 

30 

21 

JK-Total 

9 

7 

73 

74 

66 

67 

60 

58 

38 

37 

30 

29 

43 

44 

Note:  LVI  setwise  deletion  samples;  LVII  predictor/criterion  setwise  deletion 

sample.  Adjusted  for  shrinkage  (Rozeboom  formula  8).  Numbers  in  brackets 
are  the  numbers  of  predictor  scores  entering  prediction  equations. 

Decimals  omitted. 

^  CTP  =  Core  Technical  Proficiency;  GSP  =  General  Soldiering  Proficiency: 

ELS  =  Effort  and  Leadership  (LVI);  AE  =  Achievement  and  Effort  (LVII); 

MPD  =  Maintaining  Personal  Discipline  (LVI);  PD  =  Personal  Discipline  (LVII); 
PFB  =  Physical  Fitness/Military  Bearing;  HO  =  Hands-On;  JK  =  Job  Knowledge 
Number  of  MOS  for  which  validities  were  computed. 
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Incremental  Validities 


Finally,  the  incremental  validity  results  for  the  LVI  and  LVII  analyses 
are  reported  in  Tables  3.20  and  3.21.  These  results  are  also  corrected  for 
range  restriction  and  adjusted  for  shrinkage.  Table  3.20  reports  the  multiple 
correlations  for  the  four  ASVAB  factor  composites  alone  (as  computed 
separately  in  each  of  the  LVI  setwise  deletion  samples  and  each  of  the  LVII 
predictor/criterion  setwise  deletion  samples),  whereas  Table  3.21  reports  the 
multiple  correlations  for  the  four  ASVAB  factors  along  with  each  set  of 
predictors  in  the  Experimental  Battery.  Numbers  underlined  in  Table  3.21 
indicate  multiple  correlations  that  are  higher  than  those  based  on  ASVAB 
alone. 


Generally,  the  results  are  very  similar  across  the  two  samples.  For 
example,  the  results  indicate  that  the  spatial  composite  contributed  a  small 
amount  of  incremental  validity  in  both  the  LVI  and  LVII  samples,  and  that  most 
of  the  other  predictors  added  less.  Once  again,  the  primary  discrepancy 
between  the  two  sets  of  results  concerns  the  ABLE  composites.  That  is, 
whereas  the  ABLE  composites  added  eight  points  to  the  prediction  of  MPO  (from 
R  =  .16  for  ASVAB  alone  to  R  =  .24  for  ASVAB  and  ABLE  together)  and  17  points 
to  the  prediction  of  PFB  (from  R  =  .15  to  R  =  .32)  in  LVI,  they  added  nothing 
to  the  prediction  of  PD  in  LVII  and  only  five  points  to  the  prediction  of  PFB 
(from  R  =  .13  to  R  =  . 18) . 


Table  3.20 


Comparison 

of  Mean 

Multiple  Correlations 

Computed  Within 

Job  for 

ASVAB 

Factors 

Within  LVI 

Setwise 

Deletion  and  LVII 

Pred i ctor/Cr i ter i on 

Setwise 

Deletion 

Samples: 

Corrected  for 

Range  Restriction 

ASVAB 

ASVAB 

ASVAB 

ASVAB 

ASVAB 

Factors 

Factors 

Factors 

Factors 

Factors 

No. 

of 

(Spatial ) 

(Computer) 

(JOB) 

(ABLE  Comp.) 

(AVOICE) 

MOS 

a 

[5] 

[12] 

[7] 

[11] 

[12] 

Criterion 

LVI 

LVII 

LVI  LVII 

LVI 

LVII 

LVI  LVII 

LVI 

LVII 

LVI  LVII 

CTP 

9 

7 

63  65 

62 

64 

63  66 

62 

67 

64  66 

GSP 

8 

6 

66  61 

65 

62 

67  60 

66 

61 

67  61 

ELS/AE 

9 

7 

37  31 

37 

24 

37  32 

36 

30 

37  30 

MPD/PD 

9 

7 

16  17 

15 

21 

15  17 

16 

15 

16  16 

PFB 

9 

7 

16  15 

19 

17 

16  16 

15 

13 

16  18 

HO-Total 

9 

7 

51  46 

50 

45 

50  44 

50 

46 

51  47 

JK-Total 

9 

7 

71  74 

71 

74 

72  74 

71 

75 

72  74 

Note:  Adjusted  for  shrinkage  (Rozeboom  formula  8).  Numbers  in  brackets  are  the 
numbers  of  predictor  scores  entering  prediction  equations.  Decimals 
omitted. 

^  Number  of  MOS  for  which  validities  were  computed. 


148 


Table  3.21 


Comparison  of  Mean  Incremental  Correlations  Over  ASVAB  Factors  Computed  Within 
Job  for  LVI  and  LVII  Samples  for  Spatial,  Computer,  JOB,  ABLE  Composites,  and 


AVOICE:  Corrected  for  Range  Restriction 


ASVAB 

Factors  (A4) 

A4  + 

A4  + 

A4  + 

A4  + 

No.  of 

+  Spatial 

Computer 

JOB 

ABLE  Comp. 

AVOICE 

MOS^ 

[5] 

[12] 

[7] 

[11] 

[12] 

Criterion  lvi  lvii 

LVI  LVII 

LVI  LVII 

LVI  LVII 

LVI  LVII 

LVI  LVII 

CTP 

9 

7 

64 

65 

61 

63 

63 

65 

61 

65 

64 

64 

GSP 

8 

6 

69 

62 

66 

62 

67 

60 

66 

60 

66 

57 

ELS/AE 

MPD/PO 

9 

9 

7 

7 

37 

15 

33 

16 

36 

15 

10 

16 

37 

12 

30 

14 

36 

24 

24 

13 

36 

11 

20 

06 

PFB 

9 

7 

15 

12 

17 

08 

17 

13 

32 

18 

15 

11 

HO-Total 

9 

7 

53 

46 

49 

43 

50 

44 

49 

43 

50 

39 

JK-Total 

9 

7 

73 

75 

71 

73 

72 

74 

71 

74 

71 

73 

Note:  LVI  setwise  deletion  sample;  LVII  predictor/criterion  setwise  deletion 

sample.  Adjusted  for  shrinkage  (Rozeboom  formula  8).  Numbers  in  brackets 
are  the  numbers  of  predictor  scores  entering  prediction  equations. 
Underlined  numbers  denote  multiple  Rs  greater  than  for  ASVAB  Factors  alone 
(as  reported  in  Table  3.20).  Decimals  omitted. 

^  Number  of  MOS  for  which  validities  were  computed. 


SUMMARY  AND  CONCLUSION 

The  preceding  analyses  of  basic  validation  results  for  the  LVII  sample 
produced  results  that  were  largely  consistent  with  those  obtained  during  the 
basic  validation  analyses  for  LVI.  In  summarizing  the  prior  validation 
results,  Oppler,  Peterson,  and  Russell  (1994)  concluded  that  the  ASVAB  was  the 
best  overall  predictor  of  first-tour  performance,  but  that  the  composite  of 
spatial  tests  provided  a  small  amount  of  incremental  validity  for  the  can  do 
criteria  (i.e..  Core  Technical  Proficiency,  General  Soldiering  Proficiency), 
and  the  ABLE  provided  larger  increments  for  two  of  the  three  "will  do" 
criteria  (Maintaining  Personal  Discipline,  and  Physical  Fitness  and  Military 
Bearing).  The  same  pattern  of  results  was  found  in  the  present  analyses. 

Furthermore,  not  only  were  the  LVII  results  similar  in  pattern  to  those 
of  the  LVI  analyses,  they  were  also  similar  in  magnitude  (with  the  notable 
exception  of  some  of  the  ABLE  validities,  which  were  substantially  lower  in 
LVII).  In  particular,  the  multiple  correlations  between  the  ASVAB  factor 
composites  and  second-tour  criterion  scores  were  rarely  more  than  three  points 
lower  than  the  multiple  correlations  between  the  ASVAB  factors  and  the 
corresponding  first-tour  criteria.  Given  the  length  of  time  between  the 
collection  of  the  first-  and  second-tour  performance  measures  (approximately 
three  years),  this  level  of  decrement  in  validities  is  remarkably  small. 
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With  regard  to  the  .ASVAB  and  options  in  using  subtest  scores  to  form 
prediction  equations,  the  LVII  results  indicate  highly  similar  results  across 
the  four  methods  examined  (i.e.,  nine  subtests  vs.  four  factor  composites  vs. 
AFQT  vs.  MOS-specific  Aptitude  Area  composites),  with  a  slight  advantage  going 
to  the  equations  using  the  four  factors.  Such  results  were  also  reported  for 
the  LVI  sample. 

Contrary  to  results  reported  in  LVI,  however,  were  the  relative 
validities  of  the  three  ABLE  scoring  options  examined.  In  LVI,  the  method 
using  factor  scores  computed  from  the  subset  of  114  ABLE  items  proved  to  have 
generally  higher  validities,  although  the  differences  were  not  larp.  In 
comparison,  the  LVII  results  indicate  that  the  validities  of  the  three  sets  of 
scores  were  very  similar,  although  those  associated  with  the  ABLE  composites 
tended  to  be  somewhat  greater  than  those  associated  with  the  two  sets  of 
factor  scores. 

Finally,  the  results  of  the  present  analyses  indicate  that  the  ASVAB  is 
an  extremely  valid  predictor  of  the  Leadership  factor  (R  =  .63  for  the  four 
ASVAB  factor  composites),  as  are  the  other  cognitive  predictors  in  the 
Experimental  Battery.  In  fact,  none  of  the  predictor  sets  (including  the  JOB, 
ABLE  composites,  and  AVOICE)  had  multiple  correlations  of  less  than  .34  with 
this  criterion.  Furthermore,  multiple  correlations  between  ASVAB  and  the  two 
modified  versions  of  this  score  (R  =  .51  for  LDR2,  and  R  =  .47  for  LDR3) 
indicate  that  the  relationships  between  the  ASVAB  and  the  Leadership  factor 
cannot  be  explained  purely  as  being  due  to  shared  "written  verbal"  method 
variance:  The  only  "paper-and-penci 1 "  component  of  the  Leadership  factor--the 
Situational  Judgment  Test--was  not  used  in  computing  LDR2  or  LDR3. 
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Chapter  4 

THE  PREDICTION  OF  FUTURE  PERFORMANCE 
FROM  CURRENT  PERFORMANCE  AND  FROM  TRAINING  PERFORMANCE 

John  P.  Campbell,  Norman  G.  Peterson,  and  Jeffrey  Johnson 


The  general  question  of  how  accurately  individual  job  performance  at 
one  level  in  the  organization  predicts  job  performance  at  another  level  is 
virtually  a  "classic  problem"  in  personnel  research.  It  has  critical  implica¬ 
tions  for  personnel  management  as  well.  That  is,  to  what  extent  should 
selection  for  a  different  job  or  promotion  to  a  higher  level  position  be  based 
on  an  evaluation  of  an  individual's  performance  in  the  previous  job,  as 
compared  to  alternative  types  of  information  that  might  be  relevant?  In  the 
Army  context,  it  is  a  question  of  the  extent  to  which  promotion  or  reenlist¬ 
ment  decisions  should  be  based  on  assessments  of  prior  performance.  A  related 
issue  is  the  extent  to  which  initial  job  assignments  should  be  based  on 
assessments  of  performance  in  training. 


TWO  GENERAL  ISSUES 

This  general  question  encompasses  a  number  of  more  specific  queries 
about  the  nature  of  performance  in  specific  organizational  contexts.  Two  of 
these  questions  follow.  First,  the  degree  to  which  individual  differences  in 
future  performance  can  be  predicted  from  individual  differences  in  past 
performance  is  a  function  of  the  relative  stability  of  performance  across 
time.  Do  the  true  scores  for  individuals  change  at  different  rates  even  when 
all  individuals  are  operating  under  the  same  "treatment"  conditions?  The 
recent  arguments  over  this  question  sometimes  become  a  bit  heated  (Ackerman, 
1989;  Austin,  Humphreys,  &  Hulin,  1989;  Barrett,  Caldwell,  &  Alexander,  1985; 
Barrett  &  Alexander,  1989;  Barrett,  Alexander,  &  Doverspike,  1992;  Henry  & 
Hulin,  1987;  Hulin,  Henry,  &  Noon,  1990). 

The  second  question  concerns  whether  the  current  and  future  jobs  possess 
enough  commonality  in  their  knowledge,  skill,  and  ability  requirements  to 
produce  valid  predictions  of  future  performance  from  past  performance. 

Perhaps  the  new  job  is  simply  too  different  from  the  old  one.  For  example, 
the  degree  to  which  "managers"  should  possess  domain-specif ic ‘expertise  has 
long  been  argued.  Just  as  an  army  should  not  be  equipped  and  trained  to  fight 
only  the  last  war,  the  promotion  system  should  not  try  to  maximize  performance 
in  the  previous  job.  One  implication  of  this  issue  depends  on  whether  similar 
and  dissimilar  components  of  performance  for  the  two  jobs  can  be  identified 
and  measured.  If  they  can,  then  the  pattern  of  correlations  across 
performance  components  can  be  predicted,  and  the  predictions  evaluated. 

The  data  from  Project  A/Career  Force  (Campbell  &  Zook,  1991,  1990) 
permit  some  of  the  above  issues  to  be  addressed.  Extensive  job  analyses, 
criterion  development,  and  analyses  of  the  latent  structure  of  MOS  performance 
for  both  first  tour  and  second  tour  have  attempted  to  produce  a  complete 
specification  of  performance  at  each  level.  The  models  of  performance  for 
training  performance,  first-tour  performance,  and  second-tour  performance  that 
are  summarized  in  Chapter  1  provide  some  clear  predictions  about  the  pattern 
of  convergent  and  divergent  relationships  that  should  be  found.  The  results 
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of  the  prior  job  analyses  also  suggest  that  whiHe  the  new  NCO  (second  tour)  is 
beginning  to  acquire  supervisory  responsibilities,  there  is  a  great  deal  of 
communal ity  across  levels  in  terms  of  technical  task  responsibilities. 
Consequently,  first-  and  second-tour  performance  should  have  a  substantial 
proportion  of  common  determinants. 

The  three  samples  of  interest  are  from  the  longitudinal  cohort  that 
entered  the  Army  in  1986-87.  This  group  took  the  Experimental  Test  Battery  at 
the  start  of  basic  training.  Their  performance  in  training  was  assessed  at 
the  end  of  their  technical  training  course  and  their  job  performance  was 
assessed  approximately  12-24  months  after  they  entered  the  Army.  For  those 
who  reenlisted,  their  second-tour  performance  as  a  junior  NCO  was  assessed 
during  the  second  half  of  their  second  tour.  Sample  sizes  for  this  latter 
group  were  reduced  somewhat  when  soldiers  became  unavailable  for  testing 
during  Operation  Desert  Shield/Desert  Storm. 


SPECIFIC  OBJECTIVES 

•Within  this  context,  the  specific  questions  addressed  are  the  following: 

(1)  To  what  degree  does  an  individual's  level  of 
performance  in  a  first-tour  enlisted  position  predict 
performance  as  a  junior  NCO  during  his  or  her  second 
tour? 

(2)  To  what  degree  does  performance  in  training  predict 
subsequent  job  performance,  both  in  first-tour 
enlisted  positions  and  in  second-tour  positions? 

(3)  Given  that  performance  is  not  unidimensional,  do  the 
separately  measured  components  of  performance  exhibit 
the  appropriate  patterns  of  convergent  and  divergent 
validity  when  current  performance  is  used  to  predict 
future  performance? 

(4)  As  a  predictor  of  future  performance,  do  measures  of 
current  performance  add  variance  that  is  not  accounted 
for  by  measures  of  ability,  personality,  and 
interests? 


METHOD 


The  samples,  predictor  tests,  and  performance  measures  that  are  used  in 
this  analysis  have  been  described  in  detail  elsewhere  (Campbell  &  Zook,  1991). 
A  brief  recapitulation  follows. 


Samples 

The  results  reported  in  this  chapter  are  based  on  the  three  follow-ups 
of  enlisted  personnel  in  the  nine  Batch  A  MOS  in  the  Longitudinal  Validation 
sample  who  joined  the  Army  in  1986-87. 
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The  sdinpling  plan  itself  is  described  in  Campbell  and  Zook  (1991).  The 
nine  MOS  are  intended  to  be  representative  of  Army  enlisted  positions  and  the 
original  goal  was  to  begin  with  a  large  enough  number  of  new  accessions  in 
each  MOS  to  yield  500  soldiers  per  MOS  midway  through  the  first  tour  and  150 
soldiers  per  MOS  midway  through  the  second  tour.  Because  of  differential 
attrition  and  reenlistment  rates,  the  logistical  consequences  resulting  from 
the  Persian  Gulf  conflict,  and  constraints  on  the  number  of  data  collection 
sites  that  could  be  visited,  the  original  goals  were  not  reached  in  every 
case.  The  totals  for  each  MOS  at  each  measurement  point  are  shovyn  in  Table 
4.1.  These  are  the  maximum  possible  sample  sizes.  Because  of  missing  data 
considerations,  the  Ns  for  any  specific  analysis  will  be  smaller. 


Table  4.1 

Sample  Sizes  From  Each  Batch  A  MOS  When  Performance  was  Assessed  at  Three 
Points  in  Time  for  the  Project  A/Career  Force  LV  Sample _ _ _ 

Sample  Size 

_ (Before  Data  Editing) _ 

End-of-  First-Tour  Second-Tour 

Training  (EOT)  (LVI)  Job  (LVII)  Job 
Batch  A  MOS  _ Performance  Performance  Performance 

IIB  Infantryman 

13B  Cannon  Crewmember 

19E/K  Armor  Crewman 

31C  Single  Channel  Radio  Operator 
63B  Light-Wheel  Vehicle  Mechanic 

71L  Administrative  Specialist 

88M  Motor  Transport  Operator 

91A/B  Medical  Specialist 

95B  Mi  1 itary  Pol  ice 

Total  _ 


8,117 

909 

■  47 

4,712 

916 

80 

2,048 

1,073 

168 

667 

529 

70 

1,215 

752 

194 

1,414 

678 

157 

1,354 

682 

89 

3,218 

824 

222 

3,639 

452 

168 

!6.384 

6,815 

1,595 

The  Longitudinal  Predictor  Sample 

The  LV  sample  consisted  of  approximately  45,000  new  accessions  who  took 
the  Experimental  Predictor  Battery  during  their  first  three  days  in  the  Army. 
Of  the  total  sample,  approximately  30,000  would  enter  the  nine  Batch  A  MOS. 

The  tests  that  make  up  the  Experimental  Predictor  Battery  and  the  basic 
predictor  scores  that  were  derived  from  them  were  described  in  Chapter  1. 

The  End-of-Traininq  (EOT)  Sample 

The  EOT  sample  for  the  Batch  A  MOS  consists  of  those  individuals  who 
completed  their  Advanced  Individual  Training  (AIT)  and  from  whom  a  set  of 
of-Training  performance  measures  were  obtained  during  the  last  two  days  of  the 
training  period.  The  measures  consisted  of  the  project-developed  End-of- 
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Trdining  achievement  test  and  a  set  of  multiple  peer  ratings  on  ten  rating 
scales  dealing  with  technical  competence,  personnel  discipline,  effort  level, 
leadership  potential,  and  military  bearing.  Confirmatory  factor  analysis 
techniques  were  used  to  develop  composites  of  the  individual  measures  to 
reflect  scores  on  six  latent  factors  of  performance  in  training.  The  six 
performance  factors  and  the  individual  measures  that  represent  them  are  shown 
in  Figure  4.1.  The  development  of  this  factor  structure  is  described  in 
McCloy  and  Oppler  (1990)  and  summarized  in  Chapter  1  of  this  report. 


SCORES  BASED  ON  RATING  SCALES  (PEER  RATINGS) 

1)  Effort  and  Technical  Skill  (ETS)  Score 

Degree  of  effective  acquisition  of  technical  knowledge/sk i 1 1 
Degree  to  which  individual  demonstrates  extra  effort 

2)  Maintaining  Personal  Discipline  (MPD)  Score 

Degree  to  which  individual  adheres  to  regulations  and  orders 
Degree  to  which  individual  practices  effective  self-control 

3)  Physical  Fitness  and  Military  Bearing  (PFB) 

Degree  to  which  individual  maintains  proper  military  appearance 

Degree  to  which  individual  maintains  military  standards  of  physical 
fitness 

4)  Leadership  Potential  (LEAD) 

Degree  of  expected  leadership  effectiveness 

SCORES  BASED  ON  TRAINING  ACHIEVEMENT  TESTS 

5)  Basic  Knowledge  Score 

Items  measuring  knowledge  requirements  common  to  all  MOS 

6)  Technical  Knowledge  Score 

Items  measuring  technical  knowledge  requirements  specific  to 
each  MOS 


Figure  4.1.  Six  scores  based  on  measures  of  training  performance  obtained  at 
the  end  of  basic  and  technical  training. 
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The  Longitudinal  Validation  First-Tour  Performance  Assessment  Sample  (LVI) 

The  LVI  performance  measurement  sample  consisted  of  all  the  individuals 
in  the  original  LV  predictor  sample  who  were  available  for  performance 
assessment  12-24  months  later  at  any  one  of  15  data  collection  sites  in  the 
United  States,  Europe,  and  Korea.  For  the  Batch  A  MOS  this  was  approximately 
6,800  individuals. 

Each  person  underwent  one  full  day  of  performance  assessment.  The 
measures  included  a  hands-on  job  sample  test,  a  paper-and-penci 1  test  of  job 
knowledge,  a  number  of  administrative/archival  performance  indices,  and  a 
number  of  individual  rating  scales.  These  first-tour  job  performance  measures 
are  described  in  detail  in  Campbell  and  Zook  (1991)  and  summarized  in  Chapter 
1  of  this  report.  The  individual  measures  were  subjected  to  a  confirmatory 
modeling  analysis  (Oppler  et  al.,  1994)  and  five  simple  sum  factor  scores  were 
subseguently  used  as  the  best  representation  of  the  latent  structure  of  job 
performance  during  the  first  tour  of  duty.  The  five  performance  factors  and 
the  individual  measures  that  define  them  are  shown  in  Figure  4.2. 


1)  Core  Technical  Proficiency  (CTP) 

Hands-on  Test  -  MOS-Specific  Tasks 
Job  Knowledge  Test  -  MOS-Specific  Tasks 

2)  General  Soldiering  Proficiency  (GSP) 

Hands-On  Test  -  Common  Tasks 
Job  Knowledge  Test  -  Common  Tasks 

3)  Effort  and  Leadership  (ELS) 

Admin.  Index  -  Number  of  Awards  and  Certificates 
Army-Wide  BARS  Overall  Effectiveness  Rating  Scale 
Army-Wide  BARS  Effort/Leadership  Ratings  Factor 
Average  of  MOS  BARS  Ratings  Scales 

4)  Maintaining  Personal  Discipline  (MPD) 

Admin.  Index  -  Number  of  Articles  15  and  Flag  Actions 
Admin.  Index  -  Promotion  Grade  Deviation  Score 
Army-Wide  BARS  Personal  Discipline  Ratings  Factor 

5)  Physical  Fitness  and  Military  Bearing  (PFB) 

Admin.  Index  -  Physical  Readiness  Score 
Army-Wide  BARS  Fitness/Bearing  Ratings  Factor 


Figure  4.2.  LVI  first-tour  performance  factors  and  the  basic  criterion 
scores  that  define  them. 
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The  Longitudinal  Validation  Second-Tour  Performance  Measurement  Sample  (LVI I) 

The  firicil  longitudinal  follow-up  of  the  1986-87  cohort  focused  on  the 
soldiers  in  the  Batch  A  MOS  who  reenlisted  for  a  second  tour  and  who  could  be 
located  at  any  one  of  16  data  collection  sites  at  the  time  of  the  LVI I  data 
collection.  The  total  sample  size  across  the  nine  MOS  turned  out  to  be 
approximately  1,550  individuals. 

Each  individual  was  assessed  for  one  full  day  on  the  set  of  performance 
measures  developed  for  assessing  second-tour  performance.  The  development  of 
these  measures  is  described  in  detail  in  Campbell  (1991).  As  with  the 
previous  measurement  of  the  first-tour  performance  sample,  the  individual 
performance  measures  consisted  of  a  hands-on  job  sample  test,  a  comprehensive 
job  knowledge  test,  administrative/archival  indices  of  performance,  and 
multiple  rating  scales.  In  addition,  the  assessment  measures  for  second-tour 
performance  included  a  paper-and-penci 1  test  of  situational  leadership/ 
supervisory  judgment  and  three  role-play  simulations  designed  to  measure 
certain  aspects  of  supervisory/leadership  skill  having  to  do  with  counseling 
and  training  subordinates.  The  LVI I  array  of  second-tour  performance  measures 
is  summarized  in  Chapter  1  of  this  report. 

As  described  by  Hanson  et  al.  (in  press),  the  second-tour  performance 
measures  were  subjected  to  a  confirmatory  analysis  and  a  six-factor  solution 
represented  the  best  fit  for  both  the  LVII  and  the  CVII  sample  data.  The  six 
performance  factors  and  the  measures  that  define  them  are  shown  in  Figure  4.3. 

Surrenary 

To  summarize,  after  the  initial  administration  of  the  Experimental 
Predictor  Battery,  the  sample  of  soldiers  from  the  nine  Batch  A  MOS  in  the 
1986-87  cohort  was  assessed  on  performance  (a)  at  the  end  of  training,  (b) 
toward  the  end  of  their  first  tour  of  duty,  and  (c)  for  those  who  reenlisted, 
toward  the  end  of  their  second  tour  of  duty.  All  performance  indicators  were 
based  on  extensive  job  analyses,  used  multiple  measurement  methods,  and 
attempted  to  represent  the  content  of  each  job,  or  training  experience,  as 
ful ly  as  possible. 

Extensive  confirmatory  analysis  at  each  organizational  level  yielded  a 
very  consistent  picture  of  the  latent  structure  of  performance,  both  across 
cohorts  within  levels  (i.e.,  CVI  and  LVI,  and  CVII  and  LVII)  and  across  levels 
within  cohorts  (e.g.,  EOT  vs.  LVI  vs.  LVII).  The  substantive  content,  or 
latent  structure,  of  performance  showed  a  strong  tendency  to  be  consistent 
where  it  should  be  (i.e.,  across  cohorts)  and  different  where  it  should  be 
(i.e.,  across  organizational  levels). 
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Latent  Variable 

Core  Technical  Proficiency  (CTP) 


General  Soldiering  Proficiency  (GSP) 


Achievement  and  Effort  (AE)  Adm:  Awards 

Army-Wide  BARS  ratings: 

Technical  Skill/Effort  score 
Overall  Effectiveness  rating 
MOS  BARS  ratings: 

Average  across  all  ratings 

Adm:  Discipl inary  Actions 
(reversed  score) 

Army-Wide  BARS  ratings: 
Discipline  rating  score 

Physical  Fitness  and  Military  Bearing  Adm:  Physical  Readiness  score 
(PPB)  Army-Wide  BARS  ratings: 

Fitness/Bearing  score 

Leadership  (LEAD)  Adm:  Promotion  Rate 

Army-Wide  BARS  ratings: 

Leading/Supervisory  score 
Discipline  Role  Play: 

Structure  score 
Discipline  Role  Play: 

Communication  score 
Discipline  Role  Play: 

Interpersonal  Skill  score 
Counseling  Role  Play: 

Diagnosis/Prescription  score 
Counseling  Role  Play: 

Communi cat  ion/ Interpersonal 
Skills  score 
Training  Role  Play: 

Structure 

Training  Role  Play: 

Motivation  Maintenance 
Situational  Judgment  Test: 

Total  score 


Figure  4.3.  Model  of  second-tour  NCO  performance,  based  on  LVII  sample 
performance  measures. 


Maintaining  Personal  Discipline 
(MPD) 


Scores  Loading  on  Latent  Variables 


Job-Specific  Hands-On  score 
Job-Specific  Knowledge  test  score 

General  Hands-On  score 
General  Job  Knowledge  test  score 
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Analysis  Steps 

If  the  latent  structure  of  performance  is  consistent  across  replica¬ 
tions  within  organizational  level  and  differs  as  expected  (given  the  limita¬ 
tions  of  measurement)  across  levels  (i.e.,  first  tour  vs.  second  tour),  then 
it  is  reasonable  to  expect  that  the  observed  correlations  of  performance  with 
performance,  or  performance  factors  with  performance  factors,  would  show  the 
expected  convergent  and  divergent  relationships.  That  is,  a  particular 
performance  factor  measured  at  time  one  should  have  a  higher  correlation  with 
itself  at  time  two  than  it  does  with  other  performance  factors  at  time  two. 

To  examine  these  patterns  of  correlations  for  the  Project  A/Career 
Force  Project  Longitudinal  Validation  Sample,  three  basic  intercorrelation 
matrices  were  computed. 

(1)  End-of-Training  Performance  vs.  First-Tour  Performance 
(10  X  9  matrix) 

EOT  measures: 

6  EOT  performance  factor  scores 

-  Effort  and  Technical  Skill  (ETS) 

-  Maintaining  Personal  Discipline  (MPD) 

-  Physical  Fitness  and  Military  Bearing  (PFB) 

-  Leadership  Potential  (LEAD) 

-  Basic  Knowledge  score  (Base) 

-  Technical  Knowledge  score  (Tech) 

1  The  sum  of  the  Effort  and  Technical  Skill  and  the 
Leadership  scores  (E0T:ELS) 

1  The  sum  of  the  "can  do"  factors  (EOTiTech  +  EOTiBase) 

1  The  sum  of  the  "will  do"  scores  (E0T:ETS  +  E0T:LEAD  + 

E0T:MPD  +  E0T:PFB) 

1  The  grand  sum  of  all  six  factor  scores  (EOT:  Total) 

LVI  measures: 

5  First-tour  performance  factor  scores 

-  Core  Technical  Proficiency  (CTP) 

-  General  Soldiering  Proficiency  (GSP) 

-  Effort  and  Leadership  (ELS) 

-  Maintaining  Personal  Discipline  (MPD) 

-  Physical  Fitness  and  Military  Bearing  (PFB) 

1  The  sum  of  the  "can  do"  scores  (LVI:CTP  +  LVI:GSP) 

1  The  sum  of  the  "will  do"  scores  (LVI: ELS  +  LVI:MPD  + 

LVI:PFB) 

1  The  grand  sum  of  all  five  factor  scores  (LVI:Total) 

1  The  single  scale  rating  of  NCO  potential  (LVI:NC0) 

(2)  First-Tour  Performance  vs.  Second-Tour  Performance  (9  x  11  matrix) 
LVI  measures: 

9  As  listed  above  for  Matrix  (1) 

LVI I  measures: 

6  Second-tour  performance  factor  scores 

-  Core  Technical  Proficiency  (CTP) 

-  General  Soldiering  Proficiency  (GSP) 
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-  Achievement  and  Effort  (AE) 

-  Maintaining  Personal  Discipline  (MPD) 

-  Physical  Fitness  and  Military  Bearing  (PFB) 

-  Leadership  (LEAD) 

1  The  sum  of  the  LVII  Achievement  and  Effort  score  and  the 
LVII  Leadership  score 

1  The  LVII  Leadership  score  computed  without  the  Situational 
Judgment  Test  (a  paper-and-penci 1  test;  all  other 
Leadership  components  are  from  ratings) 

1  The  sum  of  the  "can  do"  factors 
1  The  sum  of  the  "will  do"  factors 

1  The  grand  sum  of  all  six  factor  scores 

(3)  End-of-Training  Performance  vs.  Second-Tour  Performance 
(10  X  11  matrix) 

EOT  measures:  As  listed  for  matrix  (1) 

LVII  measures:  As  listed  for  matrix  (2) 


Each  matrix  was  calculated  by  computing  the  intercorrelations  within 
each  MOS  and  then  averaging  over  MOS.  All  correlations  are  corrected  for 
restriction  of  range  by  using  a  multivariate  correction  that  treated  the  six 
EOT  performance  factors  as  the  "implicit"  selection  variables  on  the  grounds 
that,  in  compari'son  to  other  incidental  selection  variables,  these  factors 
would  have  the  most  to  do  with  whether  an  individual  advanced  in  the 
organization. 

Making  the  corrections  in  this  way  means  that  the  referent  population 
consists  of  all  the  soldiers  in  the  LV  sample  who  had  completed  their  training 
course.  This  is  the  population  for  which  we  would  like  to  estimate  the 
prediction  of  performance  with  performance,  and  it  is  the  population  for  which 
the  comparison  of  the  validities  of  the  experimental  predictor  tests  and 
training  criteria  as  predictors  of  future  performance  is  the  most  meaningful. 
So  long  as  the  implicit  selection  variables  are  the  best  available 
approximation  to  the  explicit  selection  variables,  the  corrected  coefficients 
will  be  a  better  estimate  of  the  population  values  than  the  uncorrected 
coefficients,  but  they  will  still  be  an  underestimate  (Linn,  1968).  Since  the 
degree  of  range  restriction  from  EOT  to  LVI  is  not  very  great,  the  effects  of 
the  corrections  were  not  very  large. 


RESULTS 

The  correlations  of  training  performance  with  first-tour  performance 
are  shown  in  Table  4.2;  the  correlations  of  first-tour  performance  with 
second-tour  performance  are  shown  in  Table  4.3;  and  the  correlations  of 
training  performance  and  second-tour  performance  are  shown  in  Table  4.4. 

In  general,  there  are  substantial  correlations  of  performance  with 
performance.  Performance  in  training  does  predict  performance  as  a  first-tour 
job  incumbent,  and  performance  in  the  first  tour  of  duty  does  predict 
performance  in  the  second  tour  after  reenlistment.  Performance  in  training 
also  predicts  performance  during  the  second  tour,  approximately  5-6  years 


159 


Table  4.2 


Zero-Order  Correlations  of  Training  Performance  (EOT)  Variables  With  First- 
Tour  Job  Performance  (LVI)  Variables;  Weighted  Average  Across  M0$ _ 


EOT  Variables^ 


LVI  Variables  EOT 

:TECH 

EOT: BASE 

E0T:ETS 

EOT:  MPD 

E0T:PFB 

EOT: LEAD 

EOT:  ELS 

EOT: CAN 

E0T:WILL 

EOT: TOT 

Core  Technical 
Proficiency  (CTP) 

.482 

3857 

.380 

3582 

.217 

3843 

,153 

3843 

.049 

3843 

.180 

3843 

.208 

3843 

.475 

3582 

.180 

3843 

.485 

3535 

General  Soldiering 
Proficiency  (GSP) 

.493 

3857 

.452 

3582 

.230 

3843 

.171 

3843 

.043 

3843 

.162 

3843 

.203 

3843 

.526 

3582 

.181 

3843 

.534 

3535 

Effort  and 

Leadership  (ELS) 

.209 

3795 

.167 

3525 

.354 

3783 

.250 

3783 

.277 

3783 

,353 

3783 

.376 

3783 

.208 

3525 

.365 

3783 

.251 

3479 

Maintain  Personal 

Discipl ine  (MPD) 

.174 

3908 

.136 

3633 

.310 

3894 

.355 

3894 

.214 

3894 

.272 

3894 

.307 

3894 

.170 

3633 

.340 

3894 

.211 

3586 

Physical  Fitness 
and  Bearing  (PFB) 

..on 

3908 

-.016 

3633 

.262 

3894 

.127 

3894 

.444 

3894 

.308 

3894 

.307 

3894 

-.015 

3633 

.330 

3894 

.031 

3586 

”Can  Do"  Performance 
Composite  (CAN) 

.530 

3857 

.451 

3582 

.245 

3843 

.177 

3843 

.050 

3843 

.187 

3843 

.226 

3843 

.545 

3582 

,197 

3843 

.555 

3535 

"Will  Do"  Performance 
Composite  (WILL) 

.167 

3795 

.128 

3525 

.386 

3783 

.302 

3783 

.373 

3783 

.389 

3783 

.413 

3783 

.163 

3525 

All 

3783 

.216 

3479 

Total  Performance 
Composite  (TOT) 

.388 

3741 

.322 

3471 

.407 

3729 

.314 

3729 

.298 

3729 

.379 

3729 

.416 

3729 

.394 

3471 

.413 

3729 

.438 

3425 

NCO  Potential 

Rating  <Supv>  (NCO) 

,165 

3458 

.140 

3458 

.309 

3444 

.224 

3444 

.259 

3444 

.306 

3444 

.327 

3444 

.169 

3458 

.324 

3444 

.208 

3397 

Note.  Corrected  for  range  restriction.  Pairwise  Ns  are  printed  below  each 
correlation.  Correlations  between  matching  variables  are  in  bold. 

^  See  full  list  in  matrix  (1)  under  "Analysis  Steps." 


later.  This  is  true  both  for  the  variables  measured  with  standardized  tests 
and  job  samples  and  for  the  variables  assessed  via  peer  ratings. 

There  is  also  a  reasonable  pattern  of  convergent  and  divergent  validity 
across  performance  factors,  even  without  correcting  these  coefficients  for 
attenuation  and  thereby  controlling  for  the  effects  of  differential 
reliability.  The  greatest  departure  from  the  expected  pattern  is  found  in  the 
differential  correlations  of  the  two  "can  do"  test-based  factors  (i.e.,  CTP 
and  GSP).  Current  CTP  does  not  always  correlate  higher  with  future  CTP  than 
current  GSP  correlates  with  future  CTP.  The  correlation  patterns  for  the 
"will  do"  factors,  which  are  based  largely  on  ratings,  virtually  never  violate 
the  expected  pattern,  even  when  peer  ratings  during  training  are  being 
correlated  with  supervisory  ratings  obtained  during  the  second  tour. 

The  one  possible  exception  to  the  consistent  results  for  the  "will  do" 
factors  is  the  predictability  of  the  leadership  performance  factor  for  second- 
tour  personnel.  This  component  of  NCO  performance  is  predicted  by  almost  all 
components  of  past  performance.  In  part,  this  is  because  the  LVII  Leadership 
factor  includes  the  Situational  Judgment  Test  as  one  of  its  component  parts. 
That  is,  the  correlations  of  LVII  Leadership  with  LVI  CTP  and  ELS  are  reduced 
when  the  SJT  is  removed  from  the  LVI  Leadership  score.  However,  they  do  not 
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Table  4. 3 


Zero-Order  Correlations  of  First-Tour  Job  Performance  (LVI)  Variables  With 
Second-Tour  Job  Performance  (LVII)  Variables:  Weighted  Average  Across  MOS 


LVI  Variables^ 


LVII  Variables  LVI:CTP 

LVI:GSP 

LVhELS 

LVIcMPD 

LVI:PFB 

LVI; CAN 

LVIrWILL 

LVI: TOT 

LVI;NC0 

Core  Technical 

.440 

.413 

.249 

.078 

.015 

.449 

.181 

.375 

.230 

Proficiency  (CTP) 

412 

412 

400 

413 

412 

412 

400 

397 

379 

General  Soldiering 

.511 

.569 

.219 

.085 

-.008 

.578 

.157 

.440 

.220 

Proficiency  (GSP) 

412 

412 

400 

413 

412 

412 

400 

397 

379 

Achievement  and 

.103 

.167 

.450 

.280 

.319 

.150 

,464 

.470 

.412 

Effort  (AE) 

390 

390 

377 

390 

390 

390 

377 

374 

353 

Leadership  (LEAD) 

.359 

.411 

.379 

.272 

.169 

.421 

.365 

.517 

.378 

344 

344 

333 

343 

342 

344 

333 

332 

319 

Leadership  Minus 

.264 

.310 

.372 

.249 

.233 

.321 

.380 

.467 

.398 

SJT  Score 

348 

348 

337 

347 

346  - 

348 

337 

336 

322 

Achievement , 

.275 

.335 

.471 

.292 

.264 

.336 

.459 

.620 

.444 

Effort  and  Leadership 

333 

333 

322 

332 

331 

333 

322 

321 

307 

Maintain  Personal 

-.044 

.038 

.116 

.257 

.166 

-.002 

.211 

.164 

.114 

Discipline  (MPD) 

406 

406 

393 

406 

406 

406 

393 

390 

370 

Physical  Fitness 

-.026 

-.013 

.220 

.135 

.460 

-.022 

.333 

.250 

.265 

and  Bearing  (PFB) 

392 

392 

379 

392 

392 

392 

379 

376 

356 

'’Can  Do"  Performance 

.520 

.533 

.259 

.097 

.010 

.562 

.193 

.452 

.260 

Composite  (CAN) 

412 

412 

400 

413 

412 

412 

400 

397 

379 

"Will  Do"  Performance 

.141 

.190 

.370 

.295 

.347 

.182 

.433 

.445 

.404 

Composite  (WILL) 

321 

321 

310 

320 

319 

321 

310 

309 

296 

Total  Performance 

.336 

.357 

.381 

.240 

.252 

.375 

.394 

.521 

.423 

Composite  (TOT) 

313 

313 

302 

312 

311 

313 

302 

301 

289 

Note.  Corrected  for  range  restriction.  Pairwise  Ns  are  printed  below  each 
correlation.  Correlations  between  matching  variables  are  in  bold. 

^  See  full  list  in  matrix  (1)  under  "Analysis  Steps." 


reduce  to  zero,  and  there  is  still  a  general  tendency  for  LVII  Leadership  to 
be  predicted  by  almost  all  aspects  of  LVI  performance. 

For  reference  purposes  the  intercorrelations  of  each  set  of  criterion 
factors,  averaged  over  MOS,  are  shown  in  Tables  4.5,  4.6,  and  4.7.  As  has 
been  shown  a  number  of  times  before,  the  intercorrelations  for  the  "can  do" 
factors  and  among  the  "will  do"  factors  are  relatively  high  and  the  cross¬ 
correlations  between  them  are  much  lower.  Given  the  high  intercorrelations 
for  the  "will  do"  factors,  the  convergent  and  divergent  relationship  they 
exhibit  is  even  more  striking. 
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Table  4.4 


Zero-Order  Correlations  of  Training  Performance  (EOT)  Variables  With  Second- 
Tour  Job  Performance  (LVII)  Variables:  Weighted  Average  Across  MOS _ 

_ EOT  Variables^ _ 


LVII  Variables  E0T:TECH  E0T:BASE  E0T:ETS  E0T:HPD  E0T:PFB  E0T:LEAD  E0T:ELS  E0T:CAN  EOTiWlLL  EOT:TOT 


Core  Technical 
Proficiency  (CTP) 

.479 

1014 

.413 

960 

.215 

1056 

.147 

1056 

.080 

1056 

.174 

1056 

.204 

1056 

.484 

960 

.183 

1056 

.480 

936 

General  Soldiering 
Proficiency  (GSP) 

.488 

1014 

.429 

960 

.192 

1056 

.107 

1056 

.064 

1056 

.112 

1056 

.155 

1056 

.496 

960 

.139 

1056 

.489 

936 

Achievement  and 

Effort  (AE) 

.098 

946 

.151 

896 

.248 

983 

.172 

983 

.189 

983 

.238 

983 

.258 

983 

.141 

896 

.250 

983 

.165 

874 

Leadership  (LEAD) 

.322 

900 

.387 

854 

.294 

931 

.191 

931 

.152 

931 

.254 

931 

.289 

931 

.396 

854 

.264 

931 

.416 

832 

Leadership  Minus 

SJT  Score 

.202 

905 

.284 

905 

.293 

936 

.203 

936 

.187 

936 

.276 

936 

.302 

936 

.274 

905 

.284 

936 

.302 

881 

Achievement, 

Effort  and  Leadership 

.238 

856 

.310 

811 

.281 

886 

.197 

886 

.168 

886 

.258 

886 

.285 

886 

.307 

811 

.268 

886 

.328 

791 

Maintain  Personal 
Discipline  (MPD) 

.080 

1006 

.086 

953 

.210 

1045 

.260 

1045 

.162 

1045 

.210 

1045 

.224 

1045 

.091 

953 

.249 

1045 

.117 

929 

Physical  Fitness 
and  Bearing  (PFB) 

-.047 

967 

-.007 

916 

.123 

1003 

.067 

1003 

.320 

1003 

.208 

1003 

.183 

1003 

-.032 

916 

.208 

1003 

-.005 

893 

"Can  Do"  Performance 
Composite  (CAN) 

.527 

1014 

.457 

960 

.228 

1056 

.145 

1056 

.082 

1056 

.160 

1056 

.201 

1056 

.534 

960 

.181 

1056 

.530 

936 

"Will  Do"  Performance 
Composite  (WILL) 

.168 

823 

.221 

780 

.270 

852 

,218 

852 

.235 

852 

.281 

852 

.295 

852 

.215 

780 

.297 

852 

.242 

761 

Total  Performance 
Composite  (TOT) 

.375 

805 

.386 

762 

.301 

831 

.225 

831 

.208 

831 

.273 

831 

,303 

831 

.417 

762 

.297 

831 

.434 

743 

Note.  Corrected  for  range  restriction.  Pairwise  Ns  are  printed  below  each 
correlation.  Correlations  between  matching  variables  are  in  bold. 

^  See  full  list  in  matrix  (1)  under  "Analysis  Steps." 


To  address  the  issue  of  whether  information  about  past  performance 
contributes  unique  variance  to  the  prediction  of  future  performance  over  that 
contained  in  measures  of  ability,  personality,  and  interests,  two  types  of 
hierarchical  regressions  were  carried  out.  The  first  sequence  specified  that 
the  order  of  entry  would  be  past  performance  first,  followed  by  the  four 
factor  scores  from  the  ASVAB,  and  then  followed  by  the  eight  composite  scores 
from  the  AVOICE,  and  then  the  seven  factor  scores  from  the  ABLE,  in  that 
order.  The  second  sequence  was  similar  to  the  first  except  that  the  order  of 
past  performance  and  ASVAB  was  reversed. 
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Table  4.5 


Intercorrelations  of  Training 
Weighted  Average  Across  MOS 

Performance  (EOT)  Variables  in 

the  LVI 

Sample: 

E0T:TECH 

E0T;BASEC 

EOTrETS 

EOTiMPD 

E0T:PFB 

EOT: LEAD 

Technical  Total  Score 

1.000 

3911 

Basic  Total  Score 

.425 

1.000 

3911 

3911 

Effort  and  TEchnical 

.185 

.174 

l.OOO 

Skill  (ETS) 

3844 

3844 

3897 

Maintain  Personal 

.143 

.121 

.678 

1.000 

Discipline  (MPD) 

3844 

3844 

3897 

3897 

Physical  Fitness 

-.011 

.039 

.626 

.436 

1.000 

and  Bearing  (PFB) 

3844 

3844 

3897 

3897 

3897 

Leadership  Potential 

.142 

.112 

.760 

.605 

.657 

1.000 

(LEAD) 

3844 

3844 

3897 

3897 

3897 

3897 

Note.  Corrected  for 

range  restriction. 

Pairwise  Ns 

are  printed  below  each 

correlation. 

Table  4.6 

Intercorrelations  of  First-Tour 
Sample:  Weighted  Average  Across 

Job  Performance 
MOS 

(LVI)  Variables  in  the 

LVI 

LVI:CTP 

LVI:GSP 

LVI:ELS 

LVI:MPD 

LVI:PFB 

LVI:  Core  Technical  Proficiency  (CTP) 

1.000 

3910 

LVI:  General  Soldiering  Proficiency  (GSP) 

.590 

3910 

1.000 

3910 

LVI:  Effort  and  Leadership  (ELS) 

.294 

3793 

.282 

3793 

1.000 

3847 

LVI:  Maintain  Personal  Discipline  (MPD) 

.212 

3907 

.218 

3907 

.615 

3847 

1.000 

3961 

LVI:  Physical  Fitness  and  Bearing  (PFB) 

.069 

3907 

.050 

3907 

.503 

3847 

.370 

3958 

1.000 

3897 

Note.  Corrected  for  range  restriction.  Pairwise  Ns  are  printed  below  each 
correlation. 
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Table  4.7 


Intercorrelations  of  Second-Tour 
Samole:  Weighted  Average  Across 

Job  Performance 
MOS 

(LVII) 

Variables  in  the 

LVII 

LVII:CTP 

LVII:GSP 

LVIIiAE 

LVIIrLDR 

LVII:MP0 

LVIzPFB 

LVII:  Core  Technical  Proficiency 

1.000 

1458 

LVII:  General  Soldiering  Proficiency 

.544 

1458 

1.000 

1458 

LVII:  Achievement  and  Effort  (AE) 

.299 

1312 

.276 

1312 

1.000 

1366 

LVII:  Leadership  (LDR) 

.467 

1237 

.496 

1237 

.544 

1208 

1.000 

1263 

LVII:  Maintain  Personal  Discipline  (MPD) 

.139 

1386 

.152 

1386 

.536 

1366 

.411 

1263 

1.000 

1443 

LVII:  Physical  Fitness  and  Bearing  (PFB) 

.094 

1339 

.091 

1339 

.432 

1318 

.289 

1223 

.359 

1392 

1.000 

1392 

Note.  Corrected  for  range  restriction.  Pairwise  /Vs  are  printed  below  each 
correlation. 


The  results  of  such  a  hierarchical  analyses  for  the  prediction  of 
first-tour  performance  from  training  performance  and  the  prediction  of  second- 
tour  performance  from  first-tour  performance  are  shown  in  Tables  4.8  through 
4.11. 

The  general  findings  illustrated  in  these  tables  seem  relatively  clear. 
Both  past  performance  and  measured  abilities  add  unique  variance  to  the 
prediction  of  future  performance.  Knowledge  of  past  performance  adds 
relatively  more,  in  comparison  to  trait  measures,  to  the  prediction  of  the 
"will  do"  components  of  future  performance  than  to  the  task  performance,  or 
"can  do"  components.  Conversely,  the  cognitive  ability  measure  (ASVAB)  adds 
more  to  the  prediction  of  future  performance  on  the  "can  do"  factors  while  the 
ABLE  adds  relatively  more  to  the  prediction  of  the  "will  do"  components. 


SUMMARY  CONCLUSIONS 

Even  though  the  intercorrelations  of  the  performance  factors  identified 
at  each  of  these  organizational  levels  are  reasonably  high,  the  results  of  the 
above  analyses  suggest  considerable  construct  validity  for  the  measures  of  the 
factors.  Past  performance  does  predict  future  performance,  and  it  does  so 
with  a  considerable  degree  of  convergent,  divergent  validity  across  the  major 
components  of  performance. 
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Table  4.8 


Multiple  Correlations  for  Predicting  First-Tour  Job  Performance  (LVI) 
Criterion  Factors  From  Training  Criteria  and  Predictor  Measures  When  the 
Training  Performance  (EOT)  Variable  is  Matched  With  its  Job  Performance 
Counterpart,  and  Incremental  Validities  Obtained  Where  ASVAB  Factors  (A4), 
AVOICE.  and  ABLE  are  Added:  Weighted  Average  Across  MOS _ 


First-Tour  Job 

Performance  Factor  (LVI) 

EOT 

[If 

EOT  +  A4 

[sr 

EOT  +  A4  + 
AVOICE 

[isr 

EOT  +  A4  + 
AVOICE  +  ABLE 
[20f 

Core  Technical  Proficiency 
(CTP) 

.489 

.538 

(.536) 

.539 

(.534) 

.541 

(.533) 

General  Soldiering 
Proficiency  (GSP) 

.451 

.577 

(.576) 

.579 

(.575) 

.584 

(.577) 

Effort  and  Leadership 
(ELS) 

.379 

(.378) 

.416 

(.412) 

.425 

(,416) 

.431 

(.419) 

Maintaining  Personal 
Discipline  (MPO) 

.356 

.370 

(.366) 

.378 

(.369) 

.400 

(.386) 

Physical  Fitness  and 

Bearing  (PFB) 

.444 

.455 

(.452) 

.465 

(.458) 

.492 

(.483) 

Note.  Corrected  for  range  restriction.  Numbers  in  brackets  are  the  number  of 
predictor  scores  entering  prediction  equations.  Numbers  in  parentheses 
are  multiple  Rs  adjusted  for  shrinkage  (Rozeboom  Formula  8). 

N  =  3,523.  ... 

®  Each  job  performance  factor  was  matched  with  its  corresponding  training 
performance  factor  except  for  LVI:ELS,  which  was  matched  with  the  weighted 
sum  of  EOTiEffort  and  Technical  Skill  and  EOT: Leadership  Potential. 
Therefore,  the  number  of  predictors  entering  the  equations  for  LVI:ELS  is 
the  number  in  brackets  +  1. 
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Table  4.9 


Multiple  Correlations  for  Predicting  First-Tour  Job  Performance  (LVI) 

Criterion  Factors  From  ASVAB  (A4)  and  Incremental  Validities  Obtained  When  the 
Appropriate  EOT  Performance  Factor,  AVOICE,  and  ABLE  are  Added:  Weighted 
Average  Across  MOS _ _ _ _ _ 

A4  + 


First-Tour  Job 
Performance 

Factor  (LVI) 

A4 

[4] 

A4  + 

EOT 

[5r 

A4  + 
AVOICE 
[12] 

A4  + 
AVOICE 
+  EOT 

[13^ 

A4  + 
AVOICE 
+  ABLE 
[19] 

AVOICE 
+  ABLE 
+  EOT 

[20r 

Core  Technical 
Proficiency  (CTP) 

.467 

(.464) 

.538 

(.536) 

.471 

(.464) 

.539 

(.534) 

.477 

(.467) 

.541 

(.533) 

General 

Soldiering 
Proficiency  (GSP) 

.542 

(.540) 

.577 

(.576) 

.545 

(.540) 

.579 

(.575) 

.552 

(.545) 

.584 

(.577) 

Effort  and 
Leadership  (ELS) 

.248 

(.243) 

.416 

(.412) 

.269 

(.255) 

.425 

(.416) 

.298 

(.279) 

.431 

(.419) 

Maintaining 

Persona! 

Discipline  (MPD) 

.148 

(.139) 

.370 

(.366) 

.165 

(.141) 

.378 

(.369) 

.264 

(.242) 

.400 

(.386) 

Physical  Fitness 
and  Bearing  (PFB) 

.149 

(.140) 

.455 

(.452) 

.203 

(.184) 

.465 

(.458) 

.339 

(.323) 

.492 

(.483) 

Note.  Corrected  for  range  restriction.  Numbers  in  brackets  are  the  number  of 
predictor  scores  entering  prediction  equations.  Numbers  in  parentheses 
are  multiple  Rs  adjusted  for  shrinkage  (Rozeboom  Formula  8). 

/V  =  3,523.  .  .  • 

^  Each  job  performance  factor  was  matched  with  its  corresponding  training 
performance  factor  except  for  LVI:ELS,  which  was  matched  with  the  weighted 
sum  of  E0T:Effort  &  Technical  Skill  and  EOT:Leadership  Potential. 

Therefore,  the  number  of  predictors  entering  the  equations  for  LVIiELS  is 
the  number  in  brackets  +  1. 
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Table  4. 10 


Multiple  Correlations  for  Training  Performance  (EOT)  Variables  Matched  With 
Second-Tour  Job  Performance  (LVII)  Variables,  and  Incremental  Validities  When 
ASVAB  Factors  (A4),  AVOICE,  and  ABLE  are  Added:  Weighted  Average  Across  MOS 


Second-Tour  Job 

Performance  Factor  (LVII) 

EOT 

[ir 

EOT  +  A4 

[5r 

EOT  +  A4 
+  AVOICE 

[isr 

EOT  +  A4 
+  AVOICE 
+  ABLE 
[20]" 

Core  Technical  Proficiency 
(CTP) 

.482 

.557 

(.553) 

.569 

(.559) 

.571 

(.555) 

General  Soldiering 

Proficiency  (GSP) 

.418 

.596 

(.593) 

.606 

(.597) 

.610 

(.596) 

Effort  and  Achievement  + 
Leadership  (ELS) 

.387 

(.384) 

.434 

(.426) 

.446 

(.429) 

.471 

(.448) 

Achievement  and  Effort 
(AE) 

.250 

.292 

(.281) 

.305 

(.279) 

.330 

(.291) 

Leadership  (LDR) 

.257 

.498 

(.493) 

.514 

(.501) 

.536 

(.518) 

Maintaining  Personal 

Discipline  (MPD) 

.261 

.272 

(.260) 

.300 

(.272) 

.324 

(.285) 

Physical  Fitness  and  Bearing 
(PFB) 

.321 

.333 

(.324) 

.371 

(.350) 

.407 

(.378) 

Note.  Corrected  for  range  restriction.  Numbers  in  brackets  are  the  number  of 
predictor  scores  entering  prediction  equations.  Numbers  in  parentheses 
are  multiple  Rs  adjusted  for  shrinkage  (Rozeboom  Formula  8). 


^  Each  job  performance  factor  was  matched  with  its  corresponding  training 
performance  factor  except  for  LVIIrELS,  which  was  matched  with  the  weighted 
sum  of  E0T:Effort  and  Technical  Skill  and  EOT: Leadership  Potential. 
Therefore,  the  number  of  predictors  entering  the  equations  for  LVII:ELS  is 
the  number  in  brackets  +  1. 
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Table  4.11 

Multiple  Correlations  for  Predicting  Second-Tour  Job  Performance  (LVII) 
Criterion  Factors  From  ASVAB  (A4)  and  Incremental  Validities  Obtained  When  the 
Appropriate  EOT  Performance  Factor,  AVOICE,  and  ABLE  are  Added.  Weighted 
Average  Across  M0$ _ _ _ _ _ _ _ 


A4  + 

A4  +  AVOICE 
AVOICE  +  ABLE 

+  ABLE  +  EOT 

[19]  [20]^ 


Core  Technical 
Proficiency  (CTP) 

.522 

(.518) 

.557 

(.553) 

.537 

(.526) 

.569 

(.559) 

.539 

(.522) 

.571 

(.555) 

General  Soldiering 
Proficiency  (GSP) 

.588 

(.585) 

.596 

(.593) 

.599 

(.590) 

.606 

(.597) 

.603 

(.589) 

.610 

(.596) 

Effort  and  Achievement 
+  Leadership  (ELS) 

.366 

(.360) 

.434 

(.426) 

.384 

(.366) 

.446 

(.429) 

.425 

(.400) 

.471 

(.448) 

Achievement  and 

Effort  (AE) 

.193 

(.179) 

.292 

(.281) 

.217 

(.179) 

.305 

(.279) 

.258 

(.208) 

.330 

(.291) 

Leadership  (LDR) 

.456 

(.452) 

.498 

(.493) 

.477 

(.464) 

.514 

(.501) 

.510 

(.492) 

.536 

(.518) 

Maintaining  Personal 
Discipline  (MPD) 

.111 

(.084) 

.272 

(.260) 

.176 

(.125) 

.300 

(.272) 

.242 

(.187) 

.324 

(.285) 

Physical  Fitness  and 
Bearing  (PFB) 

.128 

(.106) 

.333 

(.324) 

.233 

(.198) 

.371 

(.350) 

.333 

(.297) 

.407 

(.378) 

Note.  Corrected  for  range  restriction.  Numbers  in  brackets  are  the  number  of 
predictor  scores  entering  prediction  equations.  Numbers  in  parentheses 
are  multiple  Rs  adjusted  for  shrinkage  (Roseboom  Formula  8). 

N  =  I ,525. 

®  Each  job  performance  factor  was  matched  with  its  corresponding  training 
performance  factor  except  for  LVIIiELS,  which  was  matched  with  the  weighted 
sum  of  E0T:Effort  and  Technical  Skill  and  EOT: Leadersh ip  Potential. 
Therefore,  the  number  of  predictors  entering  the  equations  for  LVII:ELS  is 
the  number  in  brackets  +  1. 


A4  + 

Second-Tour  Job  A4  +  A4  +  AVOICE 
Performance  Factor  A4  EOT  AVOICE  +  EOT 
(LVII)  _ [4]  [5L  [12]  [13r 
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Chapter  5 

PREDICTION  OF  FIRST-TERM  MILITARY  ATTRITION 
USING  PRE-ENLISTMENT  PREDICTORS 

Rodney  A.  McCloy  and  Ani  S.  DiFazio 


INTRODUCTION 

Soldiers  who  fail  to  complete  their  contracted  first  term  of  service 
are  costly  to  the  military.  The  cost  of  first-term  attrition  was  conserva¬ 
tively  estimated  to  be  $200  million  (in  1989  dollars)  by  Klein,  Hawes-Dawson , 
and  Martin  (1991).  Direct  costs  include  lost  investments  such  as  training  and 
recruiting  costs,  and  compensation  costs  in  the  form  of  salary  during 
enlistment  and  subsequent  unemployment  costs  (e.g.,  Laurence,  1993;  McCloy  et 
al.,  1992).  Laurence  further  noted  that  attrition  also  leads  to  significant 
indirect  costs,  both  to  the  military  (in  the  form  of  lowered  morale  and  force 
instability)  and  to  the  individual  (in  the  form  of  possibly  reduced  future 
employment  opportunities  and  earning  potential).  As  a  result,  significant 
benefits  can  accrue  from  better  understanding  the  precursors  of  attrition  and 
using  this  information  to  select  those  recruits  who  are  less  likely  to  exit 
the  military  prematurely. 

The  Services  currently  use  high  school  diploma  graduate  status  as  an 
indicator  of  a  recruit's  chances  of  completing  his  or  her  first  term,  a 
practice  spurred  by  an  initial  Air  Force  technical  report  (Flyer,  1959)  and 
since  justified  by  years  of  supporting  evidence  (e.g..  Department  of  Defense, 
1985).  As  Laurence  (1993)  stated,  the  Flyer  study  "was  only  the  first  in  a 
very  long  line  of  research  to  conclude  that  high  school  graduates  have  lower 
attrition  rates  [than  non-diploma  graduates]  .  .  .  [and]  similar  findings  have 
been  echoed  in  countless  reports  over  the  past  30  plus  years"  (p.  5). 

In  contrast,  very  little  relationship  has  been  found  between  measures  of 
cognitive  ability  and  attrition.  High  school  graduates  scoring  in  the  lowest 
part  of  the  distribution  on  the  Armed  Forces  Qualification  Test  (AFQT)  have 
lower  rates  of  attrition  than  non-graduates  in  the  uppermost  part  of  the 
distribution.  Laurence  (1993)  suggested  that  the  relationship  between  first- 
term  attrition  and  diploma  status  might  be  due  to  differences  between  the  two 
groups  on  various  non-cognitive  characteristics.  These  characteristics,  in 
turn,  are  typically  assessed  with  instruments  such  as  temperament,  interest, 
or  biodata  inventories. 

Indeed,  although  high  school  diploma  status  is  the  best  single  predictor 

of  first-term  attrition,  biodata  instruments  have  demonstrated  incremental 

validity  (e.g.,  Steinhaus,  1988;  Trent,  1993;  White,  Nord,  Mael,  &  Young, 

1993).  Further,  research  from  the  Project  A/Career  Force  program  (e.g., 

Campbell  &  Zook,  1991,  1994a)  has  demonstrated  the  validity  of  non-cognitive 

measures  for  predicting  the  volitional,  or  "will  do,"  dimensions  of  Army  job 

performance  (e.g.,  effort,  physical  fitness  and  military  bearing,  personal 

discipline)  --  dimensions  that  may  impact  attrition. 

* 

Objectives 

The  present  research  has  two  primary  goals.  The  first  is  to  determine 
the  relationship  between  first-term  attrition  and  the  three  non-cognitive 
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predictor  measures:  the  Assessment  of  Background  and  Life  Experiences  (ABLE), 
the  Army  Vocational  Interest  Career  Examination  (AVOICE),  and  the  Job 
Orientation  Blank  (JOB).  A  critical  research  question  concerns  the 
incremental  contributions  of  these  measures  to  the  prediction  of  attrition 
over  the  information  provided  by  the  Armed  Services  Vocational  Aptitude 
Battery  (ASVAB)  and  high  school  diploma  graduate  (HSDG)  status. 

The  second  goal  is  to  use  all  the  available  predictor  data,  as  described 
above  to  develop  a  specific  predictor  composite  for  attrition  that  could  be 
used  to  select  applicants  who  have  higher  probabilities  for  completing  their 
first  term  of  service. 

The  relationship  between  first-term  attrition  and  the  pre-enlistment 
predictors  was  addressed  using  proportional  hazards  modeling  (Cox,  1972),  a 
form  of  event  history  analysis  (cf.  Allison,  1984).  Event  history  analysis 
allows  a  researcher  to  model  whether  an  event  occurs,  and  if  so,  when  it 
occurs.  In  the  present  research,  the  event  is  attrition  from  the  Army  during 
the  first  term  of  enlistment.  Specifically,  a  proportional  hazards  model 
allows  one  to  determine  the  relationship  between  one  or  more  predictor 
variables  and  the  rate  at  which  events  occur  over  time. 

Analytic  Difficulties  With  Event  Data 

Researchers  who  describe  events  as  dependent  variables  must  be  aware 
that  event  data  bring  with  them  certain  analytic  difficulties  (Allison,  1984; 
Singer  &  Willett,  1991).  Three  problems  frequently  appear.  First,  the 
presence  of  time-varying  independent  variables  often  complicates  the  analysis 
process  (Allison,  1984,  pp.  10-11).  Time-varying  independent  variables  do  not 
appear  in  every  event  study,  but  it  is  often  sensible  to  consider  them,^given 
that  the  observation  of  events  naturally  occurs  over  time,  and  a  person's 
status  on  many  potential  predictors  may  be  expected  to  change  in  a  meaningful 
way  during  the  elapsed  time. 

Second,  in  the  present  research  some  soldiers  leave  for  reasons  other 
than  those  defined  as  first-term  attrition  (e.g.,  soldiers  going  to  Officer 
Candidate  School).  These  soldiers  did  not  experience  the  event  during  the 
time  they  were  observed,  but  they  were  not  observed  for  the  entire  first  term. 
Such  observations  are  said  to  be  "censored." 

There  are  actually  several  types  of  censoring  (cf.  Lawless,  1982).  The 
censoring  in  the  above  example  is  typically  termed  "right  censoring.  As  time 
moves  left  to  right  on  a  time  line,  a  person's  observation  period  runs  along 
the  time  line  until  it  hits  a  border  on  the  right  side:  the  point  where  the 
observation  period  ends.  Any  events  or  non-events  after  (i.e.,  to  the  right 
of)  this  demarcation  are  unknown  to  the  researcher  (i.e.,  they  are  censored). 

Two  types  of  right-censored  observations  occur  in  the  current  data. 
First,  those  soldiers  who  complete  their  term  of  enlistment  are  right-censored 
at  t  =  3  (or  4)  years.  One  cannot  be  sure  that  these  soldiers  did  not  leave 
the  military  two  years,  two  months,  or  even  two  days  after  the  end  of  the 
contracted  time,  which  may  or  may  not  be  the  end  of  their  first  term  (given 
the  possibility  of  contract  extensions).  All  that  can  be  said  is  that  they 
did  not  exit  prematurely  during  the  first  term.  Second,  soldiers  who  exited 
the  Army  for  reasons  not  defined  as  attrition  (e.g..  Officer  Candidate  School) 
are  also  right-censored,  but  at  the  point  they  exited  the  enlisted  ranks. 
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Similar  to  those  soldiers  who  did  not  attrit,  all  that  can  be  said  of  these 
soldiers  is  that  they  had  not  yet  exited  prematurely  as  of  time  t  <  3  (or  4) 
years  (the  end  of  the  contracted  enlistment).  The  question  is  how  to  handle 
the  censored  observations.  Failure  to  treat  censored  observations 
appropriately  can  lead  to  biased  results. 

Third,  it  is  clear  that  the  distribution  of  the  events  over  time  in  no 
way  approximates  a  normal  distribution.  This  is  the  rule  rather  than  the 
exc6ption  for  ovont  dcitd.  HoncG,  fdoiili^r  dndlytic  tGchniquGS  bdSGd  upon 
normal  theory  are  inappropriate  for  the  analysis  of  event  data. 

Several  strategies  aimed  at  accommodating  these  characteristics  of  event 
data  have  been  tried  (cf.  Singer  &  Willett,  1991).  For  example,  one  might 
perform  an  ordinary  least-squares  (OLS)  regression  on  the  dichotomous 
dependent  variable  ATTRITION,  indicating  whether  the  event  did  or  did  not 
occur  as  of  a  specified  point  in  time  (e.g.,  t  =  2  years).  The  dichotomous 
variable  ignores  the  timing  of  the  events,  focusing  instead  on  whether  or  not 
an  event  occurred  during  the  observation  period.  Using  OLS  procedures  with  a 
dichotomous  variable  of  this  sort  creates  a  number  of  undesirable  properties 
such  as  the  possibility  of  out-of-range  predicted  values  for  the  dependent 
variable  and  violation  of  assumptions  regarding  the  variance  of  the  dependent 
variable  (e.g.,  Collett,  1991). 

A  second  option  would  be  to  ignore  the  timing  of  the  events  by 
performing  a  logistic  regression  analysis  of  the  same  dichotomous  dependent 
variable.  This  type  of  analysis  avoids  many  of  the  pitfalls  of  the  OLS 
approach  (e.g.,  the  predicted  values  of  the  criterion  will  not  lie  outside 
acceptable  bounds)  and  is  reasonable  if  (a)  few  individuals  exit  the  study  and 
(b)  the  timing  of  event  occurrence  holds  little  interest  for  the  researcher. 
Nevertheless,  the  approach  does  not  easily  accommodate  time-varying 
independent  variables  or  censored  observations.  Individuals  who  exited  the 
study  typically  will  be  deleted  from  this  design  because  they  do  not  qualify 
as  an  event,  and  labeling  them  as  a  non-event  is  inaccurate.  Clearly,  the 
greater  the. number  of  exiting  observations,  the  greater  the  potential  for  bias 
in  the  parameter  estimates  of  the  regression  model  if  they  are  excluded  from 
the  analysis.  (In  the  present  data,  the  number  of  mid-term  censored 
observations  is  quite  small,  suggesting  that  the  potential  for  bias  in  a 
logistic  regression  analysis  is  likely  to  be  small.) 

If  the  occurrence  of  events  over  time  is  of  interest,  a  second  strategy 
is  to  model  the  time  to  the  event  by  changing  the  dependent  variable  into  a 
continuous  variable,  such  as  "days  of  service."  This  approach,  however,  also 
has  drawbacks.  Consider  the  issue  of  censoring,  in  particular.  The  exiting 
observations  again  must  be  excluded  from  the  analysis,  and  there  is  a  ceiling 
effect  on  the  dependent  variable  for  the  observations  that  carry  through  the 
entire  observation  period  without  experiencing  an  event  that  is  directly 
influenced  by  the  often  arbitrary  duration  of  the  study  period.  Further,  the 
ceiling  is  inaccurate  because  it  represents  the  time  elapsed  before  the  study 
ended  rather  than  an  event  time. 

A  class  of  methods  is  available  that  accommodates  all  of  the  issues  just 
mentioned.  These  methods  have  been  used  for  many  years  in  engineering  (e.g.. 
Lawless,  1982)  and  biostatistics  (e.g.,  Kalbfleisch  &  Prentice,  1980)  but  have 
gained  attention  only  recently  in  the  psychological  literature  (e.g.,  Fichman, 
1988,  1989;  Harrison  &  Hulin,  1989;  Morita,  Lee,  &  Mowday,  1989;  Singer  & 
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Willett,  1991).  The  methods  carry  several  labels,  including  "failure-time 
analysis"  from  the  engineering  literature  and  "survival  analysis"  from  the 
biostat istical  literature.  Perhaps  the  most  general  term  for  these  methods  is 
"event  history  analysis"  (Allison,  1984). 

Event  history  analysis  allows  a  researcher  to  model  whether  an  event 
occurs  and  if  so,  when  it  occurs.  In  many  ways,  event  history  models  have 
much  in  common  with  traditional  analytic  strategies.  For  example,  event 
history  analyses  generate  both  descriptive  and  inferential  statistical 
information.  Group  differences  in  event  occurrence  can  be  tested,  and 
statistical  models  relating  independent  variables  to  event  occurrence  can  be 
developed.  Nevertheless,  the  mathematics  of  event  history  analyses  is  more 
complex  than  the  mathematics  of  correlational  analytic  methods.  The  basic 
elements  of  event  history  models  are  described  below.  Formal  treatments  of 
the  mathematics  of  event  history  models  appear  in  Kalbfleisch  and  Prentice 
(1980)  and  Lawless  (1982). 

Functions:  The  Building  Blocks  of  Event  History  Models 

Event  history  models  involve  functions,  and  the  function  values  are 
evaluated  across  time.  The  two  primary  functions  used  in  event  history 
analyses  are  the  survivor  function  and  the  hazard  function. 

Survivor  Function 


The  survivor  function,  S(t),  describes  the  probability  that  an 
individual  will  survive  at  least  until  time  t  without  experiencing  the  event 
in  question.  S(t)  is  a  monotonic,  non-increasing  (typically  decreasing) 
function.  In  this  respect,  it  is  essentially  a  reverse  cumulative 
distribution  function,  cumulating  across  time  the  proportion  of  observations 
that  have  yet  to  experience  the  event. 

Figure  5.1  contains  a  plot  of  survival  curves  for  the  first-term 
attrition  of  high  school  graduates  and  non-graduates  who  enlisted  for  three 
years  into  the  Army  Military  Occupational  Specialty  (MOS)  IIB  (Infantryman). 
The  curves  indicate  that  the  survival  rate  for  graduates  is  much  higher  than 
for  non-graduates.  For  example,  it  takes  just  under  two  years  to  reach  the 
point  where  80  percent  of  the  original  sample  of  graduates  remain,  whereas  it 
takes  just  10  months  to  winnow  the  sample  of  non-graduates  to  80  percent  of 
its  original  size. 

Hazard  Function 


Because  S(t)  is  a  monotonical ly  non-increasing  function,  its  shape 
remains  relatively  unchanged,  regardless  of  the  rate  at  which  events  occur 
over  time.  Thus,  the  survivor  function  might  appear  relatively  uninformative, 
because  not  all  events  share  the  same  pattern  of  occurrence.  For  example,  the 
probability  of  dying  increases  with  a  function  of  time  from  about  age  30 
onward  (Kalbfleisch  &  Prentice,  1980),  whereas  the  probability  of  leaving  the 
military  increases  rapidly  during  the  first  three  months  of  service  and 
decreases  to  a  relatively  stable  rate  thereafter.  The  function  describing  the 
distribution  of  event  occurrence  across  time  is  h(t),  the  hazard  function. 

The  definition  of  the  hazard  function  depends  upon  whether  time  is  measured  in 
discrete  units  or  continuously. 
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Months  of  Service 


Figure  5  1  Survivor  curves  for  high  school  diploma  graduates  and  non¬ 
graduates  in  MOS  IIB  with  3-year  terms  of  enlistment. 


Discrete  time.  For  discrete  time,  h(t)  repi^'esents  the  probability  an 
individual  will  experience  an  event  during  a  specified  time  interval,  given 
that  the  individual  is  at  risk  for  experiencing  the  event.  Hence,  the  hazard 
is  a  conditional  density  function.  Calculation  of  the  discrete-time  hazard 
depends  upon  two  quantities:  (a)  the  number  of  individuals  who  experience  the 
event  during  the  interval,  divided  by  (b)  the  number  of  individuals  who  are  at 
risk  for  experiencing  the  event  during  the  interval  what  Allison  (1984) 
labeled  the  risk  set.  For  single  events,  the  risk  set  steadily  decreases  as 
individuals  either  experience  the  event  or  are  censored. 

The  risk  set  and  its  use  in  calculating  the  discrete  time  hazard 
demonstrate  how  an  event  history  model  makes  optimal  use  of  data  from  censored 
observations.  Consider  a  researcher  investigating  what  variables  predict  the 
event  of  divorce  within  the  first  five  years  of  marriage.  Assume  that  500 
couples  appear  in  the  original  sample  at  time  t  =  0  and  that  observations  of 
marital  status  are  made  twice  a  year.  At  the  first  observation  period  of  six 
months,  5  of  the  500  couples  will  have  divorced.  Hence,  for  the  first  six- 
month  period,  the  discrete  time  hazard  is  5/500  =  .01.  During  the  second  time 
interval,  25  couples  obtain  divorces  and  10  couples  exit  the  study  while 
married  (i.e.,  they  are  censored  observations).  The  risk  set  is  now  495 
rather  than  500,  because  the  five  couples  who  divorced  during  the  first  six 
months  are  no  longer  part  of  the  sample.  Hence,  h(t)  for  the  second  time 
interval  is  25/495  =  .05.  Note  that  the  censored  observations  contribute  to 
the  risk  set  for  interval  two  but  are  not  considered  events,  because  they  did 
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not  divorce.  For  the  third  time  interval,  however,  the  risk  set  will  be 
(495  -  35)  =  460.  Thus,  the  censored  observations  do  not  contribute  to  the 
risk  set  for  the  third  interval.  This  example  demonstrates  how  the  data  for 
censored  observations  are  used  correctly  and  optimally  in  event  history 
models,  contributing  to  the  calculation  of  the  hazard  rate  (via  the  risk  set) 
for  the  amount  of  time  the  observations  are  in  the  study. 

Discrete-time  event  history  models  provide  an  easily  understood 
interpretation  of  the  hazard  function.  In  addition,  discrete-time  models  can 
be  estimated  with  standard  logistic  regression  software  programs.  For 
discussions  and'appl ications  of  discrete-time  models,  see  Singer  and  Willett 
(1993)  and  Willett  and  Singer  (1993,  in  press). 

Continuous  time.  For  continuous  time,  h(t)  is  defined  as  a  mathematical 
limit  that  describes  the  instantaneous  rate  at  which  events  occur.  Formally, 


h  ( t) 


lim 

A  C-0 


p  {  t  ^T<  t  +  A  t 

A  t 


t) 


(1) 


where  T  is  the  time  of  the  event  (Kalbfleisch  &  Prentice,  1980,  p.  6).  The 
numerator  of  the  limit  is  the  discrete  time  hazard  rate  when  At  =  1.  For  the 
continuous  time  hazard  rate,  the  probability  is  divided  by  At,  the  length  of 
the  interval;  the  limit  is  evaluated  as  this  interval  becomes  smaller.  The 
continuous  time  hazard  is  not  a  probability,  because  it  can  take  on  values 
greater  than  one,  having  a  range  of  [0,+“)  (Allison,  1984).  The  natural 
logarithm  of  the  hazard  rate  typically  serves  as  the  dependent  variable  in 
event  history  models,  as  will  be  shown  in  a  later  section.  One  way  of ^ 
conceptualizing  the  continuous-time  hazard  rate  for  the  current  situation  is 
as  the  relative  rate  at  which  people  are  leaving  the  Army  at  any  given  point 
in  time. 

Unlike  the  survivor  function,  h(t)  can  take  any  shape.  Figure  5.2  shows 
a  plot  of  the  hazard  functions  corresponding  to  Figure  5.1.  As  mentioned 
above,  these  plots  indicate  that  the  rate  of  first-term  attrition  is  greatest 
during  the  first  six  months  of  service,  after  which  it  decreases  to  a 
relatively  steady  rate. 


^  The  survivor  and  hazard  functions  are  related.  Formally,  the  hazard 
rate  is  defined  as 
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where  f(t)  is  the  probability  density  function  of  the  event  times  and  S(t)  is 
the  survivor  function.  Note  that 

d  Sit) 


fit)  =  - 


dt 


because  S(t)  is  a  reverse  cumulative  distribution  function  (Kalbfleisch  & 
Prentice,  1980). 
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Months  of  Service 


Figure  5.2,  Hazard  curves  for  high  school  diploma  graduates  and  non-graduates 
in  MOS  IIB  with  3-year  terms  of  enlistment. 


Descriptive  Statistical  Information 

When  conducting  correlational  analyses,  one  typically  wishes  to 
determine  the  characteristics  of  (i.e.,  describe)  the  sample.  The  most  common 
means  of  doing  this  is  to  calculate  the  means,  standard  deviations,  ranges, 
and  so  on  of  the  measures  administered  to  that  sample.  In  event  history 
analysis,  descriptive  information  is  provided  by  plots  of  the  survivor  and 
hazard  functions,  which  may  be  obtained  for  the  total  sample  or  for  subgroups. 
Corresponding  information  on  the  number  of  events  that  occurred  for  the 
subgroup  and  the  number  of  censored  observations  is  also  useful. 

Another  common  step  in  correlational  analyses  is  to  test  for  mean 
differences  on  one  or  more  measures  between  various  groups  in  the  analysis 
sample.  A  similar  strategy  applies  to  event  history  analyses,  but  rather  than 
testing  subgroup  differences  on  specific  measures,  the  subgroup  survivor 
functions  are  compared  to  see  if  the  event  behavior  differs  by  subgroup.  For 
example,  the  survivor  functions  in  Figure  5.1  could  be  compared  to  determine 
if  the  groups'  survival  behavior  differs  (and  it  does  at  <  .0001).  Subgroup 
differences  may  also  be  investigated  while  controlling  for  the  effects  of  one 
or  more  variables  the  investigator  thinks  might  be  related  to  survival  times. 

The  differences  in  the  survivor  functions  are  tested  using  versions  of 
log-rank  tests,  the  most  commonly  reported  being  the  Mantel-Cox  (or  Savage) 
statistic,  the  generalized  Wilcoxon  (or  Breslow)  statistic,  and  the  Tarone- 
Ware  statistic.  These  log-rank  statistics  essentially  test  the  difference 
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between  the  observed  number  of  events  in  each  of  the  groups  and  the  number  of 
events  that  would  be  expected  if  the  survivor  functions  of  all  the  groups  were 
equivalent.  The  various  statistics  differ  primarily  in  the  manner  in  which 
they  weight  observations.  For  example,  all  observations  are  weighted  equally 
in  the  calculation  of  the  Mantel-Cox  statistic,  whereas  earlier  observations 
receive  greater  weight  than  later  observations  for  the  generalized  Wilcoxon 
statistic,  thus  giving  less  weight  to  later  events  when  fewer  individuals 
remain  in  the  risk  set. 

Log-rank  tests  are  asymptotically  distributed  as  with  (g-1)  degrees 
of  freedom,  where  g  is  the  number  of  groups.  Hence,  their  use  in  small 
samples  should  be  approached  with  caution. 

Inferential  Statistical  Information 

After  examining  the  descriptive  information,  one  may  wish  to  develop 
predictive  models  of  the  dependent  variable(s)  of  interest.  For  normally 
distributed  continuous  variables,  multiple  regression  analysis  can  be  used  to 
generate  prediction  equations  relating  one  or  more  predictors  to  the 
criterion.  These  equations  can  be  used  to  obtain  predicted  scores  on  the 
criterion  for  each  observation  having  predictor  data. 

A  similar  modeling  strategy  can  be  used  in  event  history  analysis,  but 
the  criterion  variable,  method  of  estimation,  fit  statistics,  and  predicted 
values  take  different  forms.  Numerous  models  are  available.  Allison  (1984) 
distinguishes  between  parametric  models,  which  impose  a  specific  distribution 
on  the  event  times;  non-parametric  models,  which  make  few  or  no  assumptions 
about  the  distribution  of  event  times;  and  semi-parametric  models,  which 
assume  a  particular  functional  form  for  the  regression  parameters  but  do  not 
impose  any  distribution  on  the  event  times.  Detailed  information  on  various 
parametric  models  is  available  in  Kalbfleisch  and  Prentice  (1980)  and  Lawless 
(1982).  In  this  section,  the  semi-parametric  proportional  hazards  model  (Cox, 
1972)  will  be  discussed. 

The  Proportional  Hazards  Model 

Since  its  introduction  by  Cox  in  1972,  the  proportional  hazards  model 
has  become  one  of  the  most  widely  used  event  history  models.  Formally,  the 
model  is: 


In  [hit)  ]  =  In  [iioiO  1  +  PX 


(2) 


where  In  is  the  natural  logarithm,  h(t)  is  the  hazard  rate,  ho(t)  is  a 
baseline  hazard  rate  that  can  take  any  form  (i.e.,  it  is  non-parametric),  p  is 
a  vector  of  regression  coefficients,  and  X  is  a  vector  of  predictor  variables. 
Thus,  the  proportional  hazards  regression  model  relates  a  set  of  predictors  to 
the  (log)  hazard  of  the  event.  Recall  that  h(t)  has  a  range  of  [0,+“). 

Taking  the  natural  logarithm  of  the  hazard  provides  an  unbounded  criterion 
variable  with  a  range  of  (-“,  +«).  The  regression  coefficients  are  estimated 
using  a  method  devised  by  Cox  called  partial  likelihood.  The  method  has  much 
in  common  with  maximum  likelihood  estimation  procedures. 
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The  model  gets  it  name  from  the  fact  that  for  any  two  individuals  with 
covariate  vectors  and  X2,  the  ratio  of  their  hazards  is  a  constant  value,  k 

(i.e.,  the  hazards  are  proportional).  That  is, 


\{t) 

h^it) 


k 


(3) 


The  proportionality  assumption  can  be  tested  for  the  hazard  functions  of  two 
groups  by  examining  the  significance  of  the  parameter  for  a  group  dummy 
variable  (8)  interacting  with  time.  The  following  models  are  calculated: 


In  [hit)  ]  =  In  [h^it]  ]  +  6 

In  [i2(t)  ]  =  In  [hjt)]  +  p^d  +  pjX 


where  h(t)  is  the  hazard  rate,  ho(t)  is  the  baseline  hazard  for  the  group(s) 
not  modeled  by  8,  and  X  is  the  interaction  between  8  and  the  log  of  the  event 
times.  If  P2  significant,  then  the  ratio  of  the  hazard  plots  is  not 
constant,  indicating  that  the  hazard  functions  of  the  group  modeled  by  8  and 
the  other  group(s)  are  not  proportional. 

There  is  no  general  consensus  about  the  importance  of  testing  the 
proportionality  assumption.  For  example.  Singer  and  Willett  (1991)  suggested 
that  the  proportional  hazards  assumption  is  virtually  never  justified  and  its 
plausibility  should  always  be  tested.  In  contrast,  Allison  (1984)  stressed 
that  the  concept  of  proportional  hazards  is  not  an  essential  component  of  the 
model,  given  that  (a)  the  hazards  fail  to  be  proportional  as  time-varying 
independent  variables  enter  the  model,  and  (b)  even  when  the  assumption  is 
violated,  the  model  often  provides  reasonable  results.  He  argued  that 
researchers  would  better  spend  their  time  reducing  the  likelihood  of 
specification  error  in  their  models  than  worrying  about  possible  violations  of 
the  proportionality  assumption. 

If  the  hazards  prove  not  to  be  proportional,  a  stratified  analysis  may 
be  conducted  where  "group"  is  the  stratifying  variable.  The  baseline  hazards 
are  allowed  to  differ  for  each  group,  but  the  regression  parameters  are 
assumed  to  be  the  same  across  groups.  That  is. 

Group  1:  In  [hit)  ]  =  In  [h^^it)  ]  +  ,5. 

Group  2:  In  [h(t)  ]  =  In  [h^2^^'>  ^  Pi^i  ^ 


hni(t) 


and  hQ2(t)  are  group-specific  hazards  that  are  not  proportional  to 


where 

each  ottier.'  This^U  somewhk  analogous  to  estimating  regression  equations 
that  allow  group  intercepts  to  differ  while  constraining  the  slopes  to  be 
equal  across  groups. 


Whether  the  hazards  are  proportional  for  all  groups  or  only  within 
stratified  groups,  the  effect  of  the  independent  variables  on  the  hazard  is  to 
shift  the  baseline  hazard  up  or  down,  depending  on  the  values  of  the 


177 


variables.  More  formally,  the  effects  of  the  independent  variables  are 
multiplicative  on  the  hazard  rate.  This  may  be  seen  by  taking  the  antilog  of 
equation  2: 


hit)  =  hoiO 


(6) 


In  a  stratified  analysis,  an  individual's  hazard  is  shifted  up  or  down 
relative  to  the  baseline  hazard  for  their  group,  whence  equation  6  becomes 


h(t)  =  eP^  .  (7) 


with  i  being  one  of  g  groups  (i  =  1,  .  •  •,  g)-.  The  magnitude  the  hazard  is 
shifted  is  the  same  for  all  individuals  having  identical  X  vectors,  regardless 
of  group  membership.  The  shape  of  the  hazard  that  is  shifted,  however,  varies 
across  groups  (i.e.,  strata). 

Assessino  the  fit  of  proportional  hazards  models.  Determining  the  fit 
of  event  history  models  to  the  data  could  prove  frustrating  for  those 
accustomed  to  conventional  multiple  regression  analyses,  because  no  measure 
akin  to  the  coefficient  of  determination  (R‘^)  is  available,  although 
approximations  have  been  proposed  (e.g.,  Harrell,  1980).  Nevertheless,  a 
statistic  similar  to  the  F  test  for  the  hypothesis  that  all  the  parameter 
coefficients  are  zero  is  avai lable--the  likelihood  ratio  test.  The  likelihood 
function  Ts  computed  for  two  models:  (a)  the  model  in  question  with  a  select 
number  of  independent  variables,  or  covariates,  and  (b)  a  model  in  which  all 
parameters  are-  zero.  The  difference  in  the  logs  of  these  likelihood  values, 
multiplied  by  -2,  is  distributed  asymptotically  as  X  ,  with  degrees  of  freedom 
equal  to  the  number  of  parameters  in  the  model  containing  predictors.  To 
facilitate  the  use  of  the  likelihood  ratio  test,  most  statistical  software 
packages  provide  the  value  of  -2  Log  L  in  the  printout.  One  may  then  subtract 
these  values  for  two  nested  models. 

The  likelihood  ratio  test  can  also  be  used  to  test  whether  the  addition 
of  one  or  more  predictors  to  a  given  model  significantly  increases  the  fit  of 
the  model  to  the  data.  One  simply  multiplies  the  difference  in  the  log- 
likelihoods  for  the  two  models  by  -2.  As  with  the  null  model,  the  difference 
is  distributed  asymptotically  as  with  degrees  of  freedom  equal  to  the 
difference  in  the  number  of  parameters  between  the  two  models. 

Note  that  the  likelihood  ratio  test  can  be  applied  only  to  models  that 
are  nested  (i.e.,  they  are  subsets  of  one  another).  This  restriction, 
however,  applies  equally  to  tests  of  incremental  R  that  are  available  with 
linear  multiple  regression  models. 

Cross-validity  of  event  history  models.  Another  difficulty  stemming 
from  the  lack  of  a  fit  statistic  such  as  R^  is  that  cross-validation  does  not 
have  an  immediate  counterpart  in  event  history  analysis.  No  predicted  scores 
are  available  for  individuals  that  may  be  compared  to  actual  values.  Rather, 
the  models  generate  predicted  functions  over  time  for  each  individual,  which 
implies  that  each  individual's  predicted  "score"  depends  upon  time.  Likewise, 
no  shrinkage  formulae  are  available  for  estimating  the  performance  of  the 
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sample  coefficients  in  another  sample  or  in  the  population.  Nevertheless, 
McCloy  (1993)  has  proposed  a  split-sample  approach  similar  to  that  used  in 
double  cross-validation  for  estimating  the  cross-validity  of  event  history 
models. 


AN  APPLICATION  OF  EVENT  HISTORY  ANALYSIS 
TO  FIRST-TERM  ATTRITION 

Given  the  properties  of  event  history  analysis  as  described  above,  the 
proportional  hazards  model  was  used  to  analyze  attrition  data  in  the  Project 
A/Career  Force  Longitudinal  Validation  sample.  The  specific  objective  was  to 
determine  the  extent  to  which  information  from  the  Project  A/Career  Force 
Experimental  Battery  could  add  to  the  prediction  of  attrition  over  the  full 
course  of  the  first  tour  of  duty. 


Method 


Subjects 

The  subjects  were  the  roughly  49,000  first-term  soldiers  from  the 
Project  A/Career  Force  Longitudinal  Validation  (LV)  sample  (Campbell  &  Zook, 
1994).  These  soldiers  were  administered  a  4-hour  Experimental  Predictor 
Battery  (Campbell  &  Zook,  1991)  within  2  days  of  their  entry  into  the  Army. 
The  data  were  collected  over  a  15-month  period  (20  Aug  86  through  30  Nov  87) 
at  eight  Reception  Battalions.  The  jobs  represented  by  the  21  MOS  in  the  LV 
sample  ar^ Jilted  in  Table  5.1. 


Table  5.1 

Summary  of  the  21  Army  MOS  in  the  Project  A/Career  Force  LV  Sample 


MOS  Name  of  Job  _ _ _ I^OS _ Name  of  Job 


IIB 

Infantryman^ 

55B 

Ammunition  Specialist 

12B 

Combat  Engineer^ 

63B 

Light-Wheel  Vehicle  Mechanic 

13B 

Cannon  Crewman^ 

67N 

Utility  Helicopter  Repairer 

16S 

MANPADS  Crewman® 

71L 

Administrative  Specialist 

19E^ 

M60  Armor  Crewman® 

76Y 

Unit  Supply  Specialist 

19k‘’ 

Ml  Armor  Crewman® 

88M 

Motor  Transport  Operator 

27E 

Tow/Dragon  Repairer 

91A 

Medical  Specialist 

29E 

Comm. -Electronics  Radio  Repairer 

94B 

Food  Service  Specialist 

31C 

Single  Channel  Radio  Repairer 

95B 

Mi  1 itary  Police 

51B 

54B 

Carpentry/Masonry  Specialist 

NBC  Specialist 

96B 

Intelligence  Analyst 

^  Combat  group 

^  MOS  19E  and  19K  differ  only  in  terms  of  equipment  (i.e.,  type  of  tank). 
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Data  checks  for  out-of-range  values  (e.g.,  impossible  or  incompatible 
entry  and  exit  dates)  reduced  the  sample  to  48,308.  Only  those  soldiers 
having  complete  data  on  the  pre-enlistment  measures  were  used  to  model 
attrition,  yielding  a  final  sample  size  of  31,032  soldiers. 

Job  Groups.  Rather  than  running  analyses  by  MOS,  analysts  formed  job 
groups  by  splitting  the  21  MOS  listed  in  Table  5.1  into  two  groups:  Combat 
(MOS  IIB,  12B,  13B,  16S,  19E,  and  19K)  and  Non-Combat  (all  others).  These  two 
groups  (C  and  NC)  were  subdivided  by  enlistment  terms,  . as  described  below. 

In  many  applications  of  event  history  analysis,  the  endpoints  and  length 
of  the  observation  period  are  arbitrary,  being  driven  by  convenience  or  a 
desire  to  observe  a  certain  number  of  events.  First-term  attrition,  however, 
has  clear  starting  and  ending  points,  beginning  when  the  soldier  enters  the 
military  and  ending  at  the  conclusion  of  the  enlistment  term  agreed  upon  on 
the  enlistment  contract.  Army  enlistment  terms  typically  range  from  2  to  6 
years,  with  3-year  and  4-year  terms  being  the  most  common.  The  present  study 
contains  only  those  soldiers  who  agreed  to  3-year  and  4-year  terms. 

Because  the  two  terms  of  enlistment  provide  meaningful  observation 
periods  of  different  duration,  the  two  MOS  groups  were  split  by  term,  yielding 
four  analysis  groups:  soldiers  in  combat  and  non-combat  MOS  with  3-year  and 
4-year  enlistment  terms  (C3,  C4,  NC3,  and  NC4,  respectively).  Note  that  all 
six  Combat  MOS  appear  in  both  enlistment  term  groups  (C3  and  C4)  because  they 
contain  many  soldiers  with  3-year  enlistment  terms  and  many  soldiers  with  4- 
year  terms.  Similarly,  three  of  the  Non-Combat  MOS  (76Y,  94B,  and  95B)  appear 
in  both  Npn-C_Qmbat  enlistment  term  groups  (NC3  and  NC4).  The  MOS  within  each 
of  the  four  job  groups,  the  sample  sizes  for  each,  the  number  and  percentage 
who  experienced  attrition,  and  the  number  and  percentage  who  were  censored 
observations  arre  given  in  Tables  5.2  through  5.5. 


Table  5.2 


Sample  Sizes, 
for  Soldiers 

,  and  Number  and  Percent  of  Attritions  and  Censored  ( 
With  3-Year  Enlistments  in  Combat  Jobs  (C3) 

Observations 

Events 

Censored  Observations 

MOS 

N 

N 

Percent 

N 

Percent 

IIB 

4,875 

1,354 

27.8 

3,521 

72.2 

12B 

1,131 

295 

26.1 

836 

73.9 

13B 

2,416 

729 

30.2 

1,687 

69.8 

16S 

410 

140 

34.1 

270 

65.9 

19E 

287 

91 

31.7 

196 

68.3 

19K 

570 

128 

22.5 

442 

77.5 

Total 

9,689 

2,737 

28.2 

6,952 

71.8 

180 


Table  5.3 


Sample  Sizes, 
for  Soldiers 

,  and  Number  and  Percent  of  Attritions  < 
With  4-Year  Enlistments  in  Combat  Jobs 

and  Censored  ' 
(C4) 

Observations 

Events 

Censored  Observations 

MOS 

N 

N 

Percent 

N 

Percent 

IIB 

4,196 

1,397 

33.3 

2,799 

66.7 

12B 

247 

73 

29.6 

174 

70.4 

13B 

1,227 

405 

33.0 

822 

67.0 

16S 

81 

16 

19.7 

65 

80.3 

19E 

123 

47 

38.2 

76 

61.8 

19K 

566 

186 

32.9 

380 

67.1 

Total 

6,440 

2,124 

33.0 

4,316 

67.0 

Table  5.4 


Sample  Sizes,  and  Number 
for  Sol cfT^s  With  3-Year 

and  Percent  of 
Enlistments  in 

Attritions 

Non-Combat 

and  Censored 
Jobs  (NC3) 

Observations 

MOS 

N 

Events 

Censored 

Observations 

N 

Percent 

N 

Percent 

54B 

566 

149 

26.3 

417 

73.7 

55B 

198 

73 

36.9 

125 

63.1 

71L 

1,341 

362 

27.0 

979 

73.0 

76Y 

518 

115 

22.2 

403 

77.8 

88M 

769 

247 

32.1 

522 

67.9 

91A 

2,685 

649 

24.2 

2,036 

75.8 

94B 

1,579 

547 

34.6 

1,032 

65.4 

95B 

2,907 

619 

21.3 

2,288 

78.7 

Total 

10,563 

2,761 

26.1 

6,952 

73.9 
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Table  5.5 


Sample  Sizes, 
for  Soldiers 

and  Number  and  Percent  of  Attritions  and  Censored  Observations 
With  4-Year  Enlistments  in  Non-Combat  Jobs  (NC4) 

MOS 

N 

Events 

Censored  Observations^ 

N 

Percent 

N 

Percent 

27E 

104 

28 

26.9 

76 

73.1 

29E 

188 

50 

26.6 

138 

73.4 

31C 

390 

143 

36.7 

247 

63.3 

51B 

319 

88 

27.6 

231 

72.4 

63B 

1,353 

419 

31.0 

934 

69.0 

67N 

277 

55 

19.9 

222 

80.1 

76Y 

874 

275 

31.5 

599 

68.5 

94B 

429 

179 

41.7 

250 

58.3 

95B 

406 

99 

24.4 

307 

75.6 

Total 

4,340 

1,336 

30.8 

3,004 

69.2 

Measures 

All  measures  were  taken  from  the  Project  A/Career  Force  data  base.  The 
predictor  measures  were  the  same  ones  used  in  the  LVI  validation  analyses. 

Attrition.  For  this  research,  attrition  is  defined  as  a  premature 
separation  from  first-term  service  for  reasons  that  might  be  viewed  negatively 
from  the  military  perspective.  Three  critical  components  of  this  definition 
require  clarification:  U)  what  is  meant  by  premature,  (b)  which  types  of 
separation  would  be  viewed  negatively  by  the  military,  and  (c)  how  to 
establish  the  time  window  of  the  first  tour. 

First,  "premature"  is  defined  as  any  length  of  service  that  is  less  than 
the  tour  to  which  recruits  obligated  themselves  at  enlistment.  Second,  as 
developed  in  another  project  sponsored  by  the  Office  of  the  Secretary  of 
Defense,  the  Compensatory  Screening  Model  (CSM)  was  used  to  group  separation 
types  into  "pejorative"  and  "non-pejorative"  categories.  Separations  that  the 
CSM  identified  as  "pejorative,"  and  that  correspond  to  Knapp's  (1993)  Army 
separation  behavior  categories  four  and  five  were  used  to  define  negative 
separations.  The  separations  that  were  used  in  this  research  are  given  in 
Table  5.6. 


^  The  CSM  is  a  selection  procedure  for  identifying  those  non-high  school 
diploma  graduates  who  are  more  likely  to  complete  at  least  two  years  of 
obligated  service  (cf.  McBride,  1993). 
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Table  5.6 

Behaviors  Treated  as  Attrition  In  the  Present  Research^ 


•  Failure  to  meet  minimum  behavioral  or  performance  criteria  (e.g.,  poor 
performance,  disciplinary  problems,  adjustment  problems) 

•  Medical  conditions  existing  prior  to  enlistment  or  which  indicate 
failure  to  conform  to  Service  requirements  (e.g.,  weight/body  fat 
standards) 

•  Erroneous  enlistment,  underage 

•  Marriage,  pregnancy,  parenthood,  family  dependency/hardship 

•  Conscientious  objector 

•  Desertion  or  imprisonment 


^  Based  on  Knapp  (1993),  Alternative  Conceptualizations  of  Turnover. 


Third,  defining  the  first-term  window  of  time  is  easy  for  soldiers  who 
did  not  reenl.ist  and  for  those  who  reenlisted  once  their  initial  term  of 
obligation  was  completed:  It  is  the  time  between  accession  and  (first) 
separation  from  the  Army.  However,  of  the  soldiers  in  the  present  dataset  who 
reenlisted,  84T2  percent  did  so  before,  rather  than  after,  their  first  term 
was  completed.  For  this  reason,  identifying  the  length  of  the  first  tour  for 
these  soldiers  is  less  clear. 

The  following  rule  was  developed  for  these  immediate  reenlistments:  (a) 
If  the  time  between  the  basic  active  service  date  (the  date  establishing  the 
beginning  of  a  soldier's  creditable  active  service;  it  is  adjusted  for  prior 
service,  AWOL,  etc.)  and  separation  from  the  reenlistment  tour  (or  30  Sep  92, 
the  final  observation  date  available  in  the  data  base)  was  at  least  as  long  as 
the  initial  enlistment  term,  then  the  first-term  window  of  time  was  set  to  the 
enlistment  term.  Of  the  10,342  soldiers  who  reenlisted  during  their  first 
term,  10,248  (99.1%)  had  their  time  window  set  to  their  enlistment  term  -- 
that' is,  their  total  time  in  the  military  exceeded  their  initial  obligation. 

(b)  For  the  remaining  94  soldiers  (0.9%)  —  those  whose  time  from  accession  to 
separation  following  reenlistment  was  less  than  the  initial  enlistment  term 
(e.g.,  a  soldier  who  initially  enlisted  for  a  three-year  tour,  reenlisted 
after  six  months,  and  then  separated  from  the  Army  after  only  two  years  of 
service)  --  the  first-term  window  was  set  to  the  time  between  accession  and 
the  reenlistment  date. 

This  treatment  was  chosen  because  we  did  not  wish  to  include  second-term 
(i.e.,  reenlistment)  time  as  first-term  time  if  the  soldier  exited  prior  to 
the  contracted  enlistment  term.  On  the  other  hand,  we  wished  to  credit  a 
soldier  for  completing  his  or  her  contracted  enlistment,  even  if  it  was 
accomplished  as  part  of  an  immediate  reenlistment. 
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High  School  Diploma  Graduate  Status  (HSDG).  A  dummy  variable  was 
corTstructed  indicating  whether  the  individual  was  a  high  school  diploma 
graduate  (HSDG=1)  or  not  (HSDG=0). 

Armed  Services  Vocational  Aptitude  Battery  (ASVAB).  The  ten  ASVAB 
subtests  were  combined  into  four  composite  scores:  Quantitative,  Speed, 
Technical,  and  Verbal  (cf.  Campbell,  1987;  Waters,  Barnes,  Foley,  Steinhaus,  & 
Brown,  1988).  These  four  composites  were  used  in  the  present  analyses. 

Assessment  of  Background  and  Life  Experiences  (ABLE).  The  ABLE,  a  199- 
item  temperament  and  biodata  inventory,  was  designed  to  predict  Army-relevant 
criteria,  including  attrition  (Hough,  Barge,  &  Kamp,  1987).  Three  response 
options  are  available  for  each  item.  The  ABLE  contains  11  substantive  (i.e., 
content)  scales  and  four  response  validity  scales.  The  current  analyses 
included  the  11  substantive  scales  and  the  response  validity  scale  measuring 
the  Social  Desirability  of  the  soldier's  responses.  The  11  substantive  scales 
are  as  follows: 


Emotional  Stability 

Self-Esteem 

Cooperativeness 

Conscientiousness 

Nondelinquency 

Traditional  Values 


Work  Orientation 
Internal  Control 
Energy  Level 
Dominance 

Physical  Condition 


Army  Vocational  Interest  Career  Examination  (AVOICE).  The  AVOICE  is  an 
interest  inventory  that  is  based  on  the  Air  Force's  Vocational  Interest  Career 
Examination  (Peterson  et  al.,  1990).  It  contains  four  sections  comprising 
lists  of  37  jobs,  110  work  tasks,  24  spare  time  activities,  and  11  subject 
areas  relevant  to  the  Army.  Respondents  are  asked  to  indicate  whether  they 
would  like  the  jobs,  work  tasks,  spare  time  activities,  and  subject  areas. 

Five  response  options,  ranging  from  "like  very  much"  to  "dislike  very  much," 
are  available  for  each  item.  Scores  on  22  AVOICE  scales  are  computed  on  the 
basis  of  responses  to  those  items. 

Eight  AVOICE  composites,  made  up  of  unit-weighted  standard  AVOICE  scale 
scores,  were  used  in  this  study.  These  composites,  which  tap  broad  clusters 
of  interests,  were  also  developed  as  part  of  the  Experimental  Predictor 
Battery  for  Project  A/Career  Force  (Campbell  &  Zook,  1990).  The  eight  AVOICE 
composites,  along  with  the  scales  that  constitute  them,  are  as  follows: 


AVOICE  Composite 


AVOICE  Scales 


Administrative 
Audiovisual  Arts 

Food  Service 
Interpersonal 
Protective  Services 


Clerical /Administrative 

Warehousing/Shipping 

Aesthetics 

Audiographics 

Drafting 

Food  Service  (Employee) 

Food  Service  (Professional) 
Leadership/Guidance 
Medical  Services 
Fire  Protection 
Law  Enforcement 
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Rugged/Outdoors 


Ski  1 led/Technical 


Structural /Machines 


Combat 

Firearms  Enthusiast 
Rugged  Individualism 
Computers 

Electronic  Communications 

Mathematics 

Science/Chemical 

Electronics 

Heavy  Construction 

Mechanics 

Vehicle/Equipment  Operator 


Job  Orientation  Blank  (JOB).  The  JOB  is  a  31- item  inventory  developed 
to  measure  job  outcome  preferences.  This  inventory  contains  a 
features;  respondents  are  asked  to  indicate  whether  or  not  they  would  like 
each  feature  in  their  ideal  jobs.  Five  response  options,  again  ranging  from 
"like  very  much"  to  "dislike  very  much,"  are  available  for  each  item.  The  JOB 
contains  six  scales:  Job  Autonomy,  Job  Routine,  Ambition,  Job  Pride,  Job 
Security/Comfort,  and  Serving  Others.  The  first  two  scales  constitute 
composites  by  themselves,  whereas  the  standard  scores  of  the  latter  four 
scales  are  summed  to  form  a  composite  labeled  High  Expectations. 

Thus,  first-term  attrition  was  modeled  using  26  pre-enlistment 
predictors,  comprising  five  variables  available  to  the  Army  at  present  and  21 
non-cognitive  variables  derived  from  measures  developed  during  Project 
A/Career  Force.  Descriptive  statistics  for  all  26  predictors  appear  in  Table 
5.7.  The  coefficient  alpha  reliability  estimates  for  the  ABLE  scales  appear 
in  Whiten(1992,  p.  26);  estimates  for  the  AVOICE  and  JOB  appear  in  Campbell 
and  Zook  (1990). 


Table  5.7 

Descriptive  Statistics  for  the  26  Pre-Enlistment  Predictors 
Measure _ _ 


by  Job  Groups 
Std.  Dev. 


Job  Group  C3  (n  =  9,690) 


High  School  Diploma  Graduate  Status  (percent)  0.83 
ASVAB:  Quantitative  100.44 
ASVAB:  Speed  106.79 
ASVAB:  Technical  130.85 
ASVAB:  Verbal  102,43 
ABLE:  Conscientiousness  36.71 
ABLE:  Cooperativeness  44.53 
ABLE:  Dominance  27.32 
ABLE:  Emotional  Stability  40.63 
ABLE:  Energy  Level  50.90 
ABLE:  Internal  Control  41.94 
ABLE:  Nondelinquency  47.54 
ABLE:  Physical  Condition  13.82 


0.37 

13.98 

11.12 

18,42 

13.40 

3.98 

4.86 
4.51 
5.34 
5.92 
4.23 
5.56 

2.87 

(Continued) 
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Table  5.7  (continued) 


Descr  i 


ptive  Statistics  for  the  26  Pre-Enlistment  Predictors  by  Job  Group^ 


Measure 


ABLE: 

ABLE: 

ABLE: 

ABLE: 

AVOICE 

AVOICE 

AVOICE 

AVOICE 

AVOICE 

AVOICE 

AVOICE 

AVOICE 

JOB: 


Self-Esteem 
Traditional  Values 
Work  Orientation 
Social  Desirability 
:  Administrative 
:  Audiovisual  Arts 
:  Food  Service 
:  Protective  Services 
:  Rugged/Outdoors 
:  Skilled/Technical 
:  Social 

:  Structural/Machines 

Autonomy 

High  Expectations 
Routine 


Job  Group  C4 


High  School  Diploma  Graduate  Status  (percent) 

ASVAB:  Quantitative 

ASVAB:  -Speed- 

ASVAB:  Technical 

ASVAB:  Verbal^ 

ABLE:  Conscientiousness 
ABLE:  Cooperativeness 
ABLE:  Dominance 
ABLE:  Emotional  Stability 
ABLE:  Energy  Level 
ABLE:  Internal  Control 
ABLE:  Nondelinquency 
ABLE:  Physical  Condition 
ABLE:  Self-Esteem 
ABLE:  Traditional  Values 
ABLE:  Work  Orientation 
ABLE:  Social  Desirability 
AVOICE:  Administrative 
AVOICE:  Audiovisual  Arts 
AVOICE:  Food  Service 
AVOICE:  Protective  Services 
AVOICE:  Rugged/Outdoors 
AVOICE:  Skilled/Technical 
AVOICE:  Social 
AVOICE:  Structural/Machines 
JOB:  Autonomy 
JOB:  High  Expectations 
JOB;  Routine 


28.89 

29.04 

45.45 

17.09 

98.82 

149.78 

97.94 

101.49 
158.33 
200.87 

98.78 

205.50 
49.75 

196.90 

50.81 


6.440) 


0.87 

106.65 
108.14 
139.02 
108.53 

36.99 

44.80 
28.00 
41.17 
51.69 
42.58 

47.80 

13.78 
29.35 
29.25 
46.12 
16.84 
96.88 

149.44 

96.61 

103.01 

163.87 

200.66 
99.46 

204.81 

50.22 

196.78 
49.04 


Std.  Dev, 

3.91 

2.87 

6.07 

3.47 

18.45 

23.38 

17.74 

16.61 

23.17 

31.47 

16.87 

30.37 

9.86 

10.01 

10.01 


0.34 

13.05 

10.79 

15.85 

11.70 

4.14 

4.96 
4.55 
5.42 
6.02 
4.22 
5.50 

2.97 
3.94 
2.86 
6.18 
3.33 

17.63 

23.00 

17.00 

15.88 

21.67 

31.02 

16.24 

29.76 

9.61 

31.05 

9.57 


(Continued; 
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Table  5.7  (continued) 

Descriptive  Statistics  for  the  26  Pre-Enlistment  Predictors  by  Job  Group 
Measure  _ _ _ 


Job  Group  NC3  (n  =  10,563) 


High  School  Diploma  Graduate  Status 

ASVAB:  Quantitative 

ASVAB:  Speed 

ASVAB:  Technical 

ASVAB:  Verbal 

ABLE:  Conscientiousness 

ABLE:  Cooperativeness 

ABLE:  Dominance 

ABLE:  Emotional  Stability 

ABLE:  Energy  Level 

ABLE:  Internal  Control 

ABLE:  Nondelinquency 

ABLE:  Physical  Condition 

ABLE:  Self-Esteem 

ABLE:  Traditional  Values 

ABLE:  Work  Orientation 

ABLE:'  Social  Desirability 

AVOICE:  Administrative 

AVOICE:  —Audiovisual  Arts 

AVOICE:  Food  Service 

AVOICE:  Prot^tive  Services 

AVOICE:  Rugged/Outdoors 

AVOICE:  Skilled/Technical 

AVOICE:  Social 

AVOICE:  Structural/Machines 

JOB:  Autonomy 

JOB:  High  Expectations 

JOB:  Routine 


(percent)  0.89  0.32 

103.00  13.26 

108.57  11.67 

128.94  18.16 

105.29  11.29 

37.02  3.99 

44.88  4.83 

27.28  4.58 

40.10  5.54 

50.72  5.88 

42.18  4.25 

48.32  5.42 

13.18  3.09 

28.81  3.89 

29.28  2.74 

45.47  •  6.03 

16.76  3.35 

•101.79  18.56 

151.06  23.44 

101.03  19.38 

100.98  18.61 

141.48  28.59 

199.25  31.02 

104.67  17.94 

191.74  33.67 

49.67  10.02 

206.14  29.89 

50.06  9.69 


Job  Group  NC4  (n  =  4,340) 


High  School  Diploma  Graduate  Status  (percent) 

ASVAB:  Quantitative 

ASVAB:  Speed 

ASVAB:  Technical 

ASVAB:  Verbal 

ABLE:  Conscientiousness 

ABLE:  Cooperativeness 

ABLE:  Dominance 

ABLE:  Emotional  Stability 

ABLE:  Energy  Level 

ABLE:  Internal  Control 

ABLE:  Nondelinquency 

ABLE:  Physical  Condition 


0.91 

104.01 

107.78 

133.33 

104.34 
36.73 
44.40 
26.82 

39.79 
50.20 

41.79 
47.85 
13.18 


0.28 

13.18 

11.41 

18.16 

11.97 

4.03 

4.85 

4.64 

5.52 

5.88 

4.22 

5.40 

3.01 


(Continued) 
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Table  5.7  (continued) 

Descriptive  Statistics  for  the  26  Pre-Enlistment  Predictors  by  Job  Group" 


Measure _ _ 

ABLE:  Self-Esteem 
ABLE:  Traditional  Values 
ABLE:  Work  Orientation 
ABLE:  Social  Desirability 
AVOICE:  Administrative 
AVOICE:  Audiovisual  Arts 
AVOICE:  Food  Service 
AVOICE:  Protective  Services 
AVOICE:  Rugged/Outdoors 
AVOICE:  Skilled/Technical 
AVOICE:  Social 
AVOICE:  Structural/Machines 
JOB:  Autonomy 
JOB:  High  Expectations 
JOB:  Routine 


Mean 

Std.  Dev 

28.53 

3.91 

29.19 

2.74 

45.21 

6.11 

16.68 

3.37 

102.80 

19.06 

149.61 

23.13 

100.28 

19.39 

97.51 

17.78 

145.72 

26.08 

201.59 

31.08 

97.56 

17.36 

207.48 

30.95 

49.71 

9.98 

200.25 

30.48 

50.57 

9.46 

^  Job  Groups:  C3,  Combat,  3-year  enlistment;  C4,  Combat,  4-year  enl istment; 
NC3,  Non-Combat,  3-year  enlistment;  NC4,  Non-Combat,  4-year  enlistment. 


Analysis 

TestinQ~the  proportionality  assumption.  As  described  in  equation  2, 
Cox's  model  allows  the  baseline  hazard  function,  ho(t),  to  take  any  form,  with 
the  ratio  of  the  hazards  for  any  two  indiyiduals  or  groups  assumed  to  be  a 
constant  value.  Before  estimating  the  proportional  hazards  regression 
equations,  an  empirical  test  of  the  proportionality  assumption  was  conducted 
for  each  MOS  within  each  of  the  four  job  groups  by  examining  the  significance 
of  the  parameter  for  a  single  MOS  dummy  variable  (D,,,os)  interacting  with  time. 
Following  the  general  form  given  in  equation  4,  the  following  models  were 
calculated: 


In  [h{t)]  -  In  [f2Q(t)  ]  +  Pi -C^wos 

In  [71(C)]  =  ln[h^{t)]  ^  *  ^2^ 


where  h(t)  is  the  hazard  rate,  \{t)  is  the  baseline  hazard  for  the  jobs  not 
modeled  by  O^qs.  and  X  is  the  interaction  between  D„os  and  the  log  of  the  event 
times.  If  P2  significant,  then  the  ratio  of  the  hazard  plots  is  not 

constant,  indicating  that  the  hazard  function  for  the  MOS  modeled  by  D„qs  is 
not  proportional  to  the  aggregate  hazard  for  the  several  other  MOS  in  the  job 
group.  In  all  instances,  the  Pj  parameter  was  significant  at  £  <  .001, 
indicating  the  proportionality  assumption  is  untenable.  The  baseline  survivor 
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and  hazard  functions  for  one  MOS  f^om  each  of  the  four  job  groups  are  given  in 
Figures  5.3  and  5.4,  respectively.^ 

Fitting  the  proportional  hazards  models.  Because  the  hazards  proved  not 
to  be  proportional,  a  stratified  analysis  was  conducted  where  "MOS"  was  the 
stratifying  variable.  Two  types  of  proportional  hazard  analyses  were 
performed  for  each  of  the  four  job  groups. 

The  first  set  of  analyses  involved  predictor  blocks  that  were  entered 
hierarchically.  The  first  block  contained  the  four  ASVAB  composites  and  the 
dummy  variable  HSDG  denoting  high  school  diploma  graduate  status  (HSDG=1  if  a 
diploma  graduate,  zero  otherwise).  This  block  comprises  the  "base"  variables 
--  the  pre-enlistment  variables  currently  available  to  the  Army,  against  which 
incremental  prediction  was  assessed.  As  such,  block  one  was  included  in  all 
models.  Blocks  two  through  four  contained  the  ABLE,  AVOICE,  and  JOB  scores, 
respectively.  The  incremental  fit  afforded  by  each  block  was  assessed 
relative  to  block  one.  Finally,  a  model  was  estimated  containing  all  four 
blocks  of  predictors. 

The  second  type  of  analysis  used  a  best  subset  selection  algorithm  in 
the  SAS  procedure  PHREG  (SAS  Institute,  1992),  in  which  the  best  j  equations 
containing  k  specified  predictors  are  given.  The  model  considered  to  be  best 
for  a  given  number  of  predictors  is  the  one  yielding  the  highest  global  score 
chi-squared  statistic.  Here,  the  best  j  =  3  equations  containing 
k  =  .  .  .  ,26  predictors  were  obtained.  The  value  for  k  is  a  function  of 

the  number  of  predictions  that  produce  statistically  significant  increments  in 
model  ftL.  LUcelihood  ratio  tests  were  calculated  to  evaluate  the  point  at 
which  additional  predictors  failed  to  increase  the  fit  of  the  model 
significantly.  Nested  equations  were  selected  from  the  best  equations  and  re¬ 
evaluated  in  PHREG  to  obtain  the  value  of  -2  Log  L  for  each  model.  Stringent 
p-values  were  selected,  due  to  the  large  samples  and  the  dependence  of  X  on 
sample  size.  The  effect  of  each  variable,  conditional  on  all  other  variables 
in  the  model,  was  also  taken  into  account  when  deriving  the  "best"  equation 
for  each  of  the  four  groups. 


^  When  the  values  of  the  predictor  variables  do  not  vary  across  time,  the 
survivor  function  can  be  written 

Sit;  x„)  =  [ 

where  is  the  vector  of  predictor  variables  for  the  n*^^  person,  and  SgCt)  is 
the  baseline  survivor  function. 


C 

So(t)  =  exp  (-|  h^iu)  du) 
0 
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Results 


Predictor-Block  Models 

The  results  for  the  five  attrition  models  involving  predictor 
blocks  for  each  of  the  four  job  groups  are  given  in  Table  5.8.  (The 
parameters  for  each  of  the  models  are  given  in  an  Appendix  available  from  the 
first  author.)  The  top  half  of  the  table  reports  the  values  of  -2  Log  L  for 
the  five  models.  (For  stratified  analyses,  the  likelihood,  L,  maximized  for  a 
job  group  is  the  product  of  each  MOS-specific  likelihood.)  The  bottom  half  of 
the  table  contains  likelihood  ratio  test  results  (based  upon  the  values  in  the 
top  half  of  the  table)  for  specific  model  comparisons. 

Consider  job  group  C3  as  an  example.  The  value  of  -2  Log  L  for 
Model  1  (the  base  variables)  is  434.02  with  5  degrees  of  freedom  (^  <  .0001). 
Clearly,  the  base  variables  provide  significant  prediction  of  first-term 
attrition  for  soldiers  with  3-year  terms  of  enlistment  in  Combat  jobs.  Model 
2  adds  the  12  ABLE  measures  to  the  base  variables.  The  value  of  -2  Log  L 
increases  to  653.08  (again,  £  <  .0001).  To  examine  whether  this  represents  a 
statistically  significant  increase  in  the  fit  of  the  model  to  the  data,  we 
calculate  the  difference  between  -2  Log  L  for  the  two  models.  Hence, 


653.08  -  434.02  =  219.06 

2 

with  (17  -  5)  =  12  degrees  of  freedom.  The  critical  value  for  X  at  £  <  .001 
is  32. 9L,_  indicating  that  the  ABLE  provides  a  significant  increase  in  model 
fit.  The  other  values  in  the  bottom  half  of  Table  5.8  were  calculated 
simi larly. 

The  results  in  Table  5.8  indicate  the  following: 

•  The  base  variables  are  significantly  related  to  first- 
term  attrition. 

•  The  ABLE  provides  significant  incremental  fit  to  the 
base  model  for  all  four  job  groups. 

•  The  AVOICE  significantly  increases  the  fit  of  the  base 
model  for  3-year  enlistments  but  not  4-year 
enlistments.  Although  the  sample  sizes  differ  by  a 
factor  of  nearly  two  to  one  (3-year  N  =  20,252  and 
4-year  N  =  10,780),  this  is  probably  not  a  power  issue, 
given  the  absolute  size  of  each  group. 

•  Except  for  NC3,  the  JOB  does  not  provide  a 
statistically  significant  increase  in  fit  over  the  base 
model . 

•  Addition  of  the  AVOICE  and  JOB  to  the  ABLE  model 
increases  the  fit  of  the  model  to  the  data  for  job 
groups  C3  and  NC3  (Model  2  vs.  Model  5);  again,  this  is 
probably  not  a  power  issue. 
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Thus,  the  non-cognitive  measures  improve  prediction  of  first-term 
attrition  over  and  above  current  pre-enlistment  information. 

Table  5.8 

Fit  Statistics  for  the  Five  Models  Involving  Predictor  Blocks  for  Predicting 
First-Term  Attrition  by  Job  Group _ _ _ _ _ _ _ 

Values  of  -2  Log  L: 

_ Job  Group^ _ _ 


Model  df  C3 _ C4 _ NC3 _ NC4 


(1) 

Base° 

5 

434 

.02 

(2) 

Base  + 

ABLE 

17 

653 

.08 

(3) 

Base  + 

AVOICE 

13 

478 

.96 

(4) 

Base  + 

JOB 

8 

444 

.08 

(5) 

Base  + 

ALL 

28 

693 

.84 

188.48 

329.72 

102.30 

330.67 

649.52 

272.58 

207.32 

352.78 

115.67 

198.97 

356.95 

105.05 

351.96 

690.04 

284 . 40 

Values  of  -2  Log  (12  -  LI): 


Model 

Comparisons 

1  vs.  2 
1  vs.  3 
1  vs.  4 

1  vs .  5 

2  vs.  5 

3  vs .  5 

4  vs.  5 


Job  Group 


df 

C3 

C4 

NC3 

NC4 

12 

219.06 

142.19 

319.80 

170.28 

8 

44.94 

18.84"' 

23.06 

13.37"' 

3 

10.06"' 

10.49"' 

27.23 

2.75"' 

23 

259.82 

163.48 

360.32 

182.10 

11 

40.76 

21.29"' 

40.52 

11.81"' 

15 

214.88 

144.64 

337.26 

168.73 

20 

249.76 

152.99 

333.09 

179.35 

All  values  significant  at  £  <  .001  except  £  <  .01  and  non-significant. 
®  C3,  Combat,  3-year  enlistment;  C4,  Combat,  4-year  enlistment; 

NC3,  Non-Combat,  3-year  enlistment;  NC4 ,  Non-Combat,  4-year  enlistment. 
Base  =  Four  ASVAB  composites  plus  high  school  diploma  graduate  status 
(HSDG). 


Best  Subset  Selection 


The  "variable  traces"  of  the  nested  models  for  the  best  subset 
selection  analyses  are  given  in  Tables  5.9  through  5.12,  along  with  the 
associated  -2  Log  L  values  and  differences  between  them.  All  likelihood  ratio 
tests  were  evaluated  as  T  with  one  degree  of  freedom.  The  tests  for  groups 
C3  and  NC3  were  evaluated  relative  to  a  critical  value  of  10.8  (£  <  .001); 
tests  for  groups  C4  and  NC4  were  evaluated  relative  to  a  critical  value  of  7.9 
(£  <  .005).  The  traces  end  at  the  point  of  the  last  significant  increase  in 
model  fit.  All  models  begin  with  k  =  3  predictors. 
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As  shown  in  Tables  5.9  through  5.12,  the  number  of  significant 
predictors  ranges  from  seven  to  eleven.  No  AVOrcE  or  JOB  scales  appear  in  any 
of  the  equations.  Although  the  equations  are  slightly  different  across  the 
job  groups,  six  variables  appear  in  each  best  model:  high  school  diploma 
graduate  status,  and  five  ABLE  scales  (Nondelinquency,  Dominance,  Physical 
Condition,  Self-Esteem,  and  Social  Desirability).  In  addition,  the  ASVAB 
Quantitative  composite  appears  in  all  but  one  best  equation  (that  for  NC4). 
Indeed,  these  seven  variables  constitute  the  best  equation  for  both  Combat 
groups. 

Table  5.9 


Variable  Traces  and  Fit  Statistics  of  the  Nested  Attrition  Prediction  Models 
for  the  Best  Subset  Selection  Analyses:  Combat,  3-Year  Job  Group  (C3) _ 


Model 

-2  Log  L 

X^(df  -  1) 

HSDG,  QUANT, ^  Nond*" 

41,353.64 

+  Dominance 

41,328.84 

24.80 

+  Physical  Condition 

41,310.58 

18.26 

+  Social  Desirability 

41,290.80 

19.78 

+  Self-Esteem 

41,277.66 

13.14 

Significant  Values,  df  =  1:  £  <  .001 

=  10.83.  n  =  9,689 

^  ASVAB  Quantitative  composite 

ABLE  Nohdel  inquency  scale 

Table  5.10 


Variable  Traces  and  Fit  Statistics  of  the  Nested  Attrition  Prediction  Models 
for  the  Best  Subset  Selection  Analyses:  Combat,  4-Year  Job  Group  (C4) _ 


Model 

-2  Log  L 

X^  (df  =  1) 

HSDG,  QUANT, ^  Nond^’ 

31,667.26 

15.90 

+  Dominance 

31,651.36 

+  Self-Esteem 

31,631.08 

20.28 

+  Social  Desirability 

31,620.74 

10.34 

+  Physical  Condition 

31,610.32  • 

10.42 

Significant  Values,  df  =  1:  £  <  .005 

*  ASVAB  Quantitative  composite 

ABLE  Nondelinquency  scale 

=  7.88.  n  =  6,440 
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Table  5.11 


Variable  Traces  and  Fit  Statistics  of  the  Nested  Attrition  Prediction  Models 
for  the  Best  Subset  Selection  Analyses:  Non-Combat,  S-Year  Job  Group  (NC3) 

Model  L  X  ~ 


HSDG,  QUANT, ^  Cond^ 

39,482.91 

71.90 

+  Nondelinquency 

39,411.01 

+  TECHNICAL 

39,387.20 

23.81 

+  VERBAL 

39,358.51 

28.69 

+  Social  Desirability 

39,325.83 

32.68 

+  Emotional  Stability 

39,302.77 

23.06 

+  Dominance 

39,269.28 

33.49 

+  Energy  Level 

39,253.28 

16.00 

+  Self-Esteem 

39,236.97 

16.31 

Significant  Values,  df  =  1:  £  <  .001  =  10.83.  n  =  10,563 

®  ASVAB  Quantitative  composite 
^  ABLE  Physical  Condition  scale 


Table  5.U 

Variable  Traces  and  Fit  Statistics  of  the  Nested  Attrition  Prediction  Models 
for  the  Best  Subset  Selection  Analyses;  Non-Combat,  4-Year  Job  Group  (NC4) 


1  Ur  Lllc  DCOU  i  i  v^i  i  u  i  j  j  . 

Model 

-2  Log  L 

X^  (df  =  1) 

HSDG,  Nond,^  Cond*’ 

16,564.75 

29.82 

+  Dominance 

16,534.93 

+  TECHNICAL 

16,518.38 

16.55 

+  VERBAL 

16,506.40 

11.98 

+  Social  Desirability 

16,491.86 

14.54 

+  Self-Esteem 

16,482.54 

9.32 

Significant  X^  Values,  df  =  1:  £  <  .005  =  7.88.  n  =  4,340 

®  ABLE  Nondelinquency  scale 
ABLE  Physical  Condition  scale 
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The  parameter  estimates  for  each  best  equation  and  their  standard  errors 
and  p-values  are  provided  in  Tables  5.13  through  5.16.  To  help  determine  the 
conditional  impact  of  each  variable  on  the  hazard  of  first-term  attrition, 
values  of  the  risk  ratio  are  also  provided.  The  risk  ratio  is  the 
exponentiated  value  of  the  regression  coefficient  (i.e.,  e  ).  These  values 
give  some  insight  into  the  effect  of  the  independent  variables  on  the 
critGrion,  subject  to  the  usucil  Ccivecits  applied  when  deterinining  the  effect  of 
a  predictor  on  a  criterion  in  conventional  multiple  regression  analyses  based 
on  the  value  of  the  regression  coefficients  (e.g.,  Darlington,  1972).  A  value 
less  than  one  indicates  a  decrease  in  the  hazard  rate  (the  regression 
coefficient  is  negative);  a  value  greater  than  one  indicates  an  increase  in 
the  hazard  rate. 

Table  5.13 


Parameters  and  Risk  Ratios  for  the  Best  Attrition  Prediction  Equation: 
Combat,  3-Year  Job  Group  (C3)  _ _ _ 


Variable 

Regression 
Coefficient  (b) 

Standard 

Error 

P 

Risk 

One-Unit 

Increase 

Ratio 

One -SO 
Increase 

HSDG 

-.705 

.045 

.0001 

.494 

-- 

QUANTITATIVE 

-.013 

.002 

.0001 

.987 

.836 

Nondel  tirquency 

-.047 

.004 

.0001 

.954 

.769 

Dominance 

.039  • 

.006 

.0001 

1.040 

1.194 

Phys.  Cond. 

-.023 

.008 

.0014 

.976 

.933 

Social  Desir. 

.030 

.007 

.0001 

1.031 

1.113 

Self-Esteem 

to 

CNJ 

o 

.007 

.0003 

.974 

.903 

Without 

With 

Covariates 

Covariates 

X  (df  = 

7) 

-2  Log  L 

41,907.32 

41,277.66 

629 

1.66  (p  = 

.0001) 

n  =  9,689 


The  parameters  in  Tables  5.13  through  5.16  were  calculated  in  the  raw 
metric  of  the  predictors.  Hence,  they  are  analogous  to  raw  coefficients.  As 
with  raw  coefficients  in  conventional  regression,  the  risk  ratio  indicates  the 
effect  on  the  hazard  associated  with  a  one-unit  increase  in  the  predictor 
variable.  For  example,  in  the  best  equation  for  job  group  C3,  the  risk  ratio 
for  Social  Desirability  of  1.031  signifies  that  a  one-unit  increase  in  the 
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Table  5.14 


Parameters  and  Risk  Ratios  for  the  Best  Attrition  Prediction  Equation: 
Combat,  4-Year  Job  Group  (C4) _ _ _ ^ _ 


Variable 

- -  - L _ i i - — 

Regression 
Coefficient  (b) 

Standard 
Error  (o.^fhC)  . 

P 

Risk 

One-Un i t 
Increase 

Ratio 

0ne~SD 

Increase 

HSDG 

-.566 

.057 

.0001 

.568 

QUANTITATIVE 

-.008 

.002 

.0001 

.992 

.902 

Nondelinquency 

-.040 

.005 

.0001 

.961 

.803 

Dominance 

.039 

.007 

.0001 

1.040 

1.196 

Self-Esteem 

-.029 

.008 

.0004 

.971 

.891 

Social  Desir. 

.026 

.008 

.0008 

1.026 

1.090 

Phys.  Cond. 

-.027 

.008 

.0012 

.973 

.923 

Without 

With 

9 

Covariates 

Covariates 

X  (df  = 

7) 

-2  Log  L 

31,927.11 

31,610.32 

316.79  (p  ^ 

=  .0001) 

n  =  6,440 


score  on  this  ABLE  scale  increases  the  hazard  rate  for  attrition  by  3.1 
percent.'* 

If  one  wishes  to  determine  the  impact  on  the  hazard  rate  of  an  increase 
of  X  units  (or  decrease  of  -x  units),  one  simply  raises  the  risk  ratio  to  the 
power  of  X  (or  -x).  For  example,  to  determine  the  increase  on  the  hazard  rate 
for  a  five-unit  increase  in  the  Social  Desirability  score,  one  computes 
(1.031)^  =  1.165.  Thus,  a  five-unit  increase  In  the  Social  Desirability  score 
increases  the  hazard  of  first-term  attrition  by  16.5  percent. 

This  same  approach  allows  one  to  determine  the  effect  of  an  increase  of 
one  standard  deviation  (a)  in  the  measures,  which  is  equivalent  to  obtaining 
standardized  regression  coefficients.  The  metrics  of  the  predictors  vary  to  a 


'*  In  the  special  case  of  dichotomous  (i.e.,  dummy)  variables,  the  risk 
ratio  describes  the  difference  in  the  hazard  rates  for  the  two  groups.  For 
example,  the  risk  ratio  for  HSDG  of  .494  suggests  that  the  hazard  rate  for 
high  school  graduates  is  49.4  percent  of  that  for  non-graduates. 
Equivalently,  the  hazard  rate  for  non-graduates  is  (1/0.494)  =  2.024,  or 

202.4  percent,  or  slightly  over  twice  that  of  high  school  graduates. 
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Table  5.15 


Parameters  and  Risk  Ratios  for  the  Best  Attrition  Prediction  Equation:  Non- 
Combat,  3-Year  Job  Group  (NC3) _ _ _ _ _ 


r  '  > 

Risk 

Ratio 

Variable 

Regression 
Coefficient  (b) 

Standard 

Error 

P 

One-Unit 

Increase 

One-SD 

Increase 

HSDG 

-.712 

.054 

.0001 

.491 

-- 

QUANTITATIVE 

-.012 

.002 

.0001 

.989 

.858 

Phys.  Cond. 

-.066 

.007 

.0001 

.936 

.816 

Nondelinquency 

-.039 

.004 

.0001 

.962 

.808 

TECHNICAL 

-.010 

.001 

.0001 

.990 

.835 

VERBAL 

.014 

.002 

.0001 

1.014 

1.165 

Social  Desir. 

-.033 

.005 

.0001 

.968 

1.128 

Emotional  Stab. 

.036 

.007 

.0001 

1.037 

.833 

Dominance 

.032 

.006 

.0001 

1.032 

1.157 

Energy  Level 

.026 

.005 

.0001 

1.027 

1.169 

Self-Esteem 

-.030 

.008 

.0001 

.970 

.887 

Without 

Covariates 

With 

Covariates 

X  (df  = 

7) 

-2  Log  L 

39,877.43 

39,236.97 

640 

.46  (p  = 

.0001) 

n  =  10,563 


considerable  degree,  even  within  the  same  instrument.  Hence,  risk  ratios 
based  upon  one-unit  increases  could  be  of  little  interest.  To  facilitate 
metric-free  comparisons,  Tables  5.13  through  5.16  also  contain  the  values  of 
the  risk  ratios  raised  to  the  power  of  a. 

The  results  indicate  that  the  largest  conditional  effects  are  quite 
similar  across  the  four  job  groups,  with  the  largest  effects  provided  by  the 
ABLE  scales  Nondelinquency  and  Dominance.  A  higher  Nondelinquency  score 
translates  into  a  decreased  hazard  for  attrition,  whereas  the  opposite  is  true 
for  Dominance.  For  example,  a  one-unit  increase  in  the  Nondelinquency  score 
for  Combat  soldiers  with  three-year  enlistments  decreases  the  hazard  for 
attrition  to  95.4  percent  of  its  current  level.  By  comparison,  a  one-standard 
deviation  increase  in  the  nondelinquency  score  results  in  a  76.9  percent 
decrease  in  the  hazard.  Equivalently  (and  perhaps  easier  to  picture),  a 
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Table  5.16 


Parameters  and  Risk  Ratios  for  the  Best  Attrition  Prediction  Equation:  Non- 

Comb  a  t^4-YearJobGrou£_JJ^^ _ _ 


Risk 

Ratio 

Variable 

Regression 
Coefficient  (b) 

Standard 

Error 

P 

One-Unit 

Increase 

One-SD 

Increase 

HSDG 

-.568 

.084 

.0001 

.567 

-- 

Nondelinquency 

-.046 

.006 

.0001 

.955 

.779 

Phys.  Cond. 

-.067 

.010 

.0001 

.935 

.816 

Dominance 

.046 

.008 

.0001 

1.047 

1.239 

TECHNICAL 

-.011 

.002 

.0001 

.989 

.824 

VERBAL 

.013 

.003 

.0001 

1.013 

1.163 

Social  Desir. 

.038 

.009 

.0001 

1.038 

1.136 

Self-Esteem 

-.031 

.010 

.0022 

.970 

.887 

Without 

Covariates 

With 

Covariates 

1  (df  = 

7) 

-2  Log  L 

16,726.49 

16,482.54 

243 

.95  (p  = 

.0001) 

n  =  4,340 

one-unit  decrease  in  the  Nondelinquency  score  yields  risk  ratios  of  1/.954  = 
1.048,  signifying  an  increase  in  the  hazard  of  4.8  percent.  A  one-standard 
deviation  decrease  would  yield  ratios  of  1/.769  =  1.300,  signifying  a  30 
percent  increase  in  the  hazard.^ 

Summary 

The  results  of  the  block  analyses  were  reasonably  consistent  across  the 
four  job/enlistment  term  groups.  The  best  subset  analyses  yielded  remarkably 
similar  results  across  the  four  groups,  identifying  six  variables  (high  school 
diploma  graduate  status.  Nondelinquency,  Dominance,  Physical  Condition,  Self- 
Esteem,  and  Social  Desirability)  that  appear  in  all  four  "best"  equations  and 
one  (the  Quantitative  composite  from  the  ASVAB)  that  appears  in  three  of  the 
four.  These  seven  variables  were  combined  to  yield  a  single  composite  for 
predicting  first-term  Army  attrition. 


^  The  distribution  of  Nondelinquency  is  negatively  skewed,  with  a  mean 
for  C3  of  47.5,  a  standard  deviation  of  5.6,  and  a  maximum  value  of  60.  Thus, 
full  standard  deviation  increases  in  the  Nondelinquency  scale  score  might  be 
difficult  to  attain  for  a  large  number  of  individuals. 
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Analyses  of  the  Attrition  Composite 

Calculatinq  reqression  weights .  Because  the  seven  variables  chosen  for 
the  attrition  composite  represent  the  best  equation  for  the  two  Combat  job 
groups,  regression  weights  are  available  for  these  variables  from  the  best 
subset'selection  analyses  for  the  Combat  groups  (see  Tables  5.13  and  5.14). 
However,  for  the  Non-Combat  job  groups,  regression  weights  for  the  seven 
predictors  were  not  available  from  the  previous  analyses.  Thus,  the  first 
step  in  analyzing  the  attrition  composite  was  to  estimate  a  proportional 
hazards  regression  model  for  job  groups  NC3  and  NC4  using  these  seven 
variables.  The  resulting  equations  are  given  in  Tables  5.17  and  5.18. 
Although  not  selected  as  the  best  equation  for  the  two  Non-Combat  job  groups, 
the  seven  variables  clearly  evidence  a  significant  relationship  with  first- 
term  attrition  in  these  groups,  as  well. 

Table  5.17 

Parameters  and  Risk  Ratios  for  the  Attrition  Composite:  Non-Combat,  3-Year 


Job  GrouD  (NC3) 

Variable 

Regression 
Coefficient  (b) 

Standard 

(f'seib)) 

P 

Risk  Ratio 

One-Unit  One-SD 

Increase  Increase 

HSDG 

-.674 

.052 

.0001 

.510 

-- 

Nondel inquency 

-.037 

.004 

.0001 

.964 

.820 

Dominance 

.032 

.006 

.0001 

1.032 

1.155 

Phys.  Cond. 

-.063 

.007 

.0001 

.939 

.823 

Self-Esteem 

-.036 

.007 

.0001 

.964 

.867 

Social  Desir. 

.035 

.007 

.0001 

1.035 

1.122 

QUANTITATIVE 

-.013 

.002 

.0001 

.987 

.841 

Without 

With 

Covariates 

Covar i ates 

X 

2  (df  = 

7) 

-2  Log  L 

39,877.43 

39,345.51 

531. 

92  (p=. 

0001) 

n  =  10,563 


Determining  effects  of  the  composite  at  various  cut  scores.  One  might 
ask  what  increase  in  survivability  would  be  attained  if  the  seven-variable 
attrition  composite  were  used  to  screen  recruits.  The  appropriate  baseline 
for  comparison  would  be  the  attrition  behavior  of  the  present  sample  of  first 
term  soldiers. 
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Table  5.18 


Parameters  and  Risk  Ratios  for  the  Attrition  Composite:  Non-Combat,  4-Year 
Job  Group  (NC4) _ _ _ _ 


Variable 

Regression 
Coefficient  (b) 

Standard 
Error  (a^e(b)) 

P 

Risk 

One-Unit 

Increase 

Ratio 

One-SD 

Increase 

HSDG 

-.572 

.083 

.0001 

.564 

-- 

Nondel inquency 

-.044 

.006 

.0001 

.957 

.788 

Dominance 

.048 

.008 

.0001 

1.049 

1.249 

Phys.  Cond. 

-.070 

.010 

.0001 

.932 

.809 

Self-Esteem 

-.030 

.010 

.0030 

.971 

.891 

Social  Desir. 

.033 

.009 

.0003 

1.034 

1.119 

QUANTITATIVE 

-.006 

.002 

.0075 

.994 

.924 

Without 

With 

Covariates 

Covariates 

X  (df  = 

7) 

-2  Log  L 

16,726.49 

16,504.01 

222.48  (p  = 

.0001) 

n  =  4,340 


To  examine  this  question,  each  person  was  scored  on  the  appropriate 
optimally  weighted  attrition  composite.  The  same  seven  variables  appear  in 
each  composite,  but  each  variable  is  weighted  by  the  regression  coefficients 
given  in  Tables  5.13,  5.14,  5.17,  and  5.18.  The  distribution  of  scores  on 
each  composite  was  analyzed  using  the  SAS  procedure  UNIVARIATE.  From  these 
analyses,  the  scores  representing  the  10th,  25th,  and  33rd  percentiles  on  the 
attrition  composite  were  identified.  Although  the  first  two  cut  scores  were 
chosen  arbitrarily,  the  33rd  percentile  was  selected  to  represent  the 
forecasted  maximum  reduction  expected  from  the  force  drawdown.  The 
composite  distribution  was  truncated  at  these  three  cut  points. 


®  Janice  Laurence,  personal  communication,  7  October  1993. 
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Descriptive  statistics  were  calculated  within  job  group  for  the 
attrition  composite  and  the  seven  component  variables  for  those  individuals 
scoring  above  each  cut  score  (see  Tables  5.19  and  5.20).  Survival  curves  were 
also  obtained.  From  the  descriptive  statistics,  the  relative  increase  in  the 
component  score  with  increasing  truncation  of  the  composite  distribution  was 
represented  by  an  effect  size, 

~ 

Effect  size  =  — - - - 


where  m^  and  mj^  are  the  means  of  the  component  in  the  truncated  and  baseline 
groups,  respectively,  and  is  the  baseline  group  standard  deviation. 

Survival  curves  were  plotted  to  indicate  the  increase  in  survival  expected  for 
those  scoring  above  the  attrition  composite  cut  scores  (see  Figures  5.5 
through  5.8). 

Table  5.19 


Descriptive  Statistics  for  the  Attrition  Composite  at  Various  Truncation 
Points  by  Job  Group _ 


Job  Group 

N 

Mean 

SD 

Minimum 

Maximum 

C3  -  Baseline 

9,690 

3.61 

.46 

1.52 

4.87 

C3  -  10%  Cut 

8,721 

3.71 

.35 

2.96 

4.87 

C3  -  25%  Cut  ■ 

7,267 

3.82 

.28 

3.34 

4.87 

C3  -  33%  Cut 

6,492 

3.86 

.26 

3.47 

4.87 

C4  -  Baseline” 

6,440 

2.95 

.37 

1.41 

3.97 

C4  -  10%  Cut 

5,796 

3.03 

.28 

2.41 

3.97 

C4  -  25%  Cut 

4,830 

3.12 

.21 

2.75 

3.97 

C4  -  33%  Cut 

4,315 

3.15 

.20 

2.85 

3.97 

NC3  -  Baseline 

10,563 

4.15 

.45 

1.90 

5.50 

NC3  -  10%  Cut 

9,506 

4.24 

.34 

3.55 

5.50 

NC3  -  25%  Cut 

7,922 

4.34 

.28 

3.89 

5.50 

NC3  -  33%  Cut 

7,077 

4.39 

.26 

3.99 

5.50 

NC4  -  Baseline 

4,340 

*  3.22 

.40 

.99 

4.33 

NC4  -  10%  Cut 

3,906 

3.30 

.30 

2.69 

4.33 

NC4  -  25%  Cut 

3,255 

3.39 

.25 

2.99 

4.33 

NC4  -  33%  Cut 

2,908 

3.43 

.23 

3.08 

4.33 

Note:  The  value  of  the  attrition  composite  is  the  sum  of  its  seven  variables 
weighted  by  their  respective  regression  coefficients  (i.e.,  2bX).  The 
magnitude  of  these  values  is  a  function  of  the  coefficients,  b,  which 
were  obtained  when  predicting  the  natural  log  of  the  hazard  function 
(i.e..  In  [h(t)])  rather  than  h(t)  itself  (cf.  equations  2  and  6)). 


Table  5.20  indicates  that,  for  each  job  group,  the  primary  means  of 
decreasing  attrition  is  to  increase  the  percentage  of  high  school  diploma 
graduates.  Although  not  particularly  surprising,  the  effect  remains  striking, 
with  at  least  98  percent  of  high  school  diploma  graduates  appearing  in  the 
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upper  75  percent  of  the  composite  distribution.  The  ABLE  scales  show 
relatively  modest  increases  in  the  truncated  groups,  with  the  largest  effect 
sizes  (0.32  to  0.37)  obtained  for  Nondelinquency  at  a  truncation  point  of  33 
percent. 

Note  that  although  Dominance  and  Social  Desirability  both  receive 
positiv6  r6gr8Ssion  wBiQhts  in  the  proportiondl  hszdrds  6C|Udtions  (signifying 
higher  scores  are  indicative  of  higher  hazards  for  attrition),  the  effect 
sizes  for  these  variables  are  positive  in  the  majority  of  job  groups  (once  for 
Dominance,  NC3;  and  all  four  times  for  Social  Desirability)  and  always  very 
small  (ranging  from  0.00  to  0.06). 

Figures  5.5  through  5.8  demonstrate  the  effects  of  the  truncation  on 
survivability.  As  a  reference  point,  these  figures  contain  lines  indicating 
the  times  at  which  each  group  would  have  experienced  attrition  of  20  percent. 
For  all  four  plots,  the  truncation  point  of  10  percent  translates  to  an 
increase  in  the  expected  survivability  of  three  to  four  months,  whereas  the 
increase  evidenced  at  25  percent  is  seven  to  eight  months.  The  maximal 
truncation  of  33  percent  affords  an  increase  in  survivability  of  between  nine 
and  ten  months,  a  relatively  small  increase  in  survival  time  over  the  more 
modest  25  percent  screen. 

A  second  way  of  looking  at  the  curves  is  in  terms  of  the  percentage  of 
the  groups  remaining  after  six  months.  Six  months  was  chosen  because  the  rate 
of  attrition  is  highest  during  the  first  six  months,  falling  off  to  a 
relatively  constant  rate  thereafter.  This  information  for  the  baseline  and 
three  truncation  points  is  given  in  Table  5.21  for  each  of  the  four  job 
groups.  Taking  a  weighted  average  of  the  increase  in  retention,  we  would 
expect  an  average  increase  in  cohort  survival  across  the  four  job  groups  of 
0.9  percent,  iTS  percent,  and  2.1  percent  after  six  months  with  truncations  of 
10  percent,  25  percent,  and  33  percent,  respectively.  Similarly,  at  the  two- 
year  point,  the  expected  gains  are  1.6  percent,  4.3  percent,  and  5.0  percent, 
respectively. 

Increased  survivability  for  diploma  graduates  and  non-graduates. 

Departing  a  bit  from  the  attrition  composite  and  focusing  on  the  five  ABLE 
scales  that  are  components,  one  might  ask  what  increases  in  survivability 
obtain  for  high  school  diploma  graduates  and  non-graduates  who  score  above  the 
mean  on  the  five  ABLE  scales  relative  to  graduates  and  non-graduates  who  score 
low  on  the  five  ABLE  scales.  The  data  in  Table  5.20  suggest  that  most  of  the 
reduction  in  attrition  would  come  from  selecting  only  high  school  diploma 
graduates.  If  so,  how  much  of  a  contribution  do  the  ABLE  scales  make  to  the 
prediction  of  attrition,  over  and  above  high  school  diploma  graduate  status? 

To  examine  this  question,  the  five  ABLE  scales  were  standardized  within 
job  group,  unit  weighted  (with  Dominance  and  Social  Desirability  weighted 
negatively),  and  summed.  This  composite  score  was  then  standardized  within 
job  group.  For  each  job  group,  high  school  diploma  graduates  and  non¬ 
graduates  were  identified  and  split  into  two  groups:  those  scoring  above  the 
mean  (ABLE  composite  >  0  --  High  ABLE)  and  those  scoring  at  or  below  the  mean 
(ABLE  composite  <0  —  Low  ABLE). 
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Table  5.21 


Percent  of  Cohort 
Composite 

Remaining  at  Various 

Levels  of  Truncation  on 

the  Attrition 

Job  Group 

Truncation 

C3 

C4 

NC3 

NC4 

Six  Months 

No  Truncation 

91.0 

92.0 

91.1 

91.8 

Lower  10% 

91.8 

92.7 

92.2 

92.8 

Lower  25% 

92.7 

93.5 

93.2 

93.6 

Lower  33% 

93.0 

93.7 

93.5 

93.9 

Two  Years 

No  Truncation 

77.6 

77.5 

79.8 

80.8 

Lower  10% 

80.0 

79.5 

82.0 

82.8 

Lower  25% 

82.2 

81.9 

83.9 

84.7 

Lower  33% 

83.0 

82.5 

84.7 

85.2 

The  number  of  soldiers  in  each  HSDG/ABLE  grouping  and  the  percentage  of 
those  who  exit  early,  for  each  of  the  job  groups,  are  given  in  Table  5.22. 

Note  that  for  all  four  job  groups,  more  graduates  score  high  than  low  on  the 
ABLE,  whereas  Yhe  opposite  is  true  for  non-graduates. 

The  effect  of  high  and  low  ABLE  scores  on  the  survivability  of  graduates 
and  non-graduates  is  most  clearly  demonstrated  by  the  survivor  functions  that 
are  given  in  Figures  5.9  through  5.12.  These  figures  show  the  strong  effect 
of  high  school  diploma  graduate  status,  with  graduates  having  higher  survival 
rates  than  non-graduates.  Nevertheless,  for  both  groups  of  soldiers,  those 
scoring  high  on  the  composite  formed  by  the  five  ABLE  scales  that  are 
components  of  the  attrition  composite  have  a  better  rate  of  survivability  than 
those  from  the  same  educational  group  who  score  low  on  the  composite. 

Perhaps  the  most  striking  feature  of  Figures  5.9  through  5.12  is  that 
there  is  a  greater  increase  in  survivability  for  those  high  on  ABLE  for 
diploma  graduates  than  for  non-oraduates.  That  is,  although  non-graduates  who 
score  high  on  the  ABLE  demonstrate  increased  survivability  over  non-graduates 
who  score  low  on  the  ABLE,  the  difference  between  survival  curves  is  much 
larger  for  graduates.  Figures  5.9  through  5.12  indicate  that  for  all  job 
groups  except  NC3,  low-scoring  non-graduates  require  approximately  one  year 
for  20  percent  of  the  group  to  exit  prematurely,  whereas  it  takes 
approximately  14  to  15  months  for  attrition  to  reach  20  percent  for  the  high- 
scoring  non-graduates.  (The  difference  for  NC3  jobs  is  approximately  seven 
months.  Note  that  the  reference  line  for  both  NC3  groups  is  at  .85, 
representing  15  percent  attrition,  because  the  high-scoring  graduates  do  not 
experience  20  percent  attrition  during  their  first  term.)  By  comparison,  the 
difference  between  high-  and  low-scoring  graduates  ranges  from  13  to  18  months 
for  the  four  job  groups. 
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Table  5.22 


Number  of  High  School  Diploma  Graduates  and  Non-Graduates  Scoring  Above  or 
Below  the  Mean  on  the  ABLE  Composite _ 


Job  Group 

Graduate 
High  ABLE 

Graduate 
Low  ABLE 

Non-Grad 
High  ABLE 

Non-Grad 
Low  ABLE 

Total 

C3 

N 

(Percent) 

4,402 

(45.4) 

3,652 

(37.7) 

604 

(6.2) 

1,031 

(10.6) 

9,689 

(100.0) 

Attritions 

(Percent) 

891 

(20.2) 

1,062 

(29.1) 

271 

(44.9) 

513 

(49.8) 

2,737 

(28.2) 

C4 

N 

(Percent) 

3,033 

(47.1) 

2,570 

(39.9) 

308 

(4.8) 

529 

(8.2) 

6.440 

(100.0) 

Attritions 

(Percent) 

779 

(25.7) 

906 

(35.3) 

154 

(50.0) 

285 

(53.9) 

2,124 

(33.0) 

NC3 

N 

(Percent) 

5,040 

(47.7) 

4,336 

(41.0) 

417 

(3.9) 

770 

(7.3) 

10,563 

(100.0) 

Attritions 

(Percent) 

937 

(18.6) 

1,279 

(29.5) 

165 

(39.6) 

380 

(49.4) 

2,761 

(26.1) 

NC4 

N 

(Percent^ 

2,095 

(48.3) 

1,875 

(43.2) 

150 

(3.5) 

220 

(5.1) 

4,340 

(100.0) 

Attritions 

(Percent^ 

473 

(22.6) 

674 

(35.9) 

76 

(50.7) 

113 

(51.4) 

1,336 

(30.8) 

This  finding  is  of  particular  note  in  light  of  the  research  that  has 
attempted  to  use  biodata  to  select  those  non-graduates  having  the  best  chance 
of  completing  their  first  term  of  service  (McBride,  1993).  Figures  5.9 
through  5.12  suggest  that  although  the  ABLE  would  be  useful  for  selecting  such 
non-graduates,  a  greater  gain  from  using  the  ABLE  would  be  realized  if  it  were 
used  as  a  screen  for  identifying  the  high  school  diploma  graduates  who  are 
less  likely  to  complete  their  first  term. 

This  finding  may  be  due  in  part  to  range  restriction  on  the  ABLE  in  the 
non-high  school  diploma  graduate  sample.  Because  non-graduates  are  more 
likely  to  exit  the  Army  prematurely,  they  are  screened  more  stringently  before 
they  are  enlisted  than  are  high  school  diploma  graduates.  For  example, 
enlisted  non-graduates  typically  must  have  high  ASVAB  scores  and  pristine 
moral  records.-  Perhaps  requiring  a  number  of  compensating  characteristics 
from  non-graduates  leads  to  a  restriction  in  individual  variability  along  the 
dimensions  assessed  by  the  ABLE.  If  so,  this  would  have  an  adverse  effect  on 
the  predictive  relationship  between  the  ABLE  and  the  attrition  behavior  of 
these  soldiers. 
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1,0 


Time  (Months) 

—  Grads,  High  ABLE  -o- Grads.  Low  ABLE  ♦Non-Grads.  High  ABLE  —  Non-Grads,  Low  ABLE. 
(0=4.402)  (0=3.652)  (o  =  604)  (o  =  1.031) 


Figure  5.9.  Survivor  functions  for  high  school  diploma  graduates  and  non-graduates 
scoring  above/below  the  mean  of  the  ABLE  composite:  Job  Group  C3. 


Figure  5.10.  Survivor  functions  for  high  school  diploma  graduates  and  non-graduates 
scoring  above/below  the  mean  of  the  ABLE  composite:  Job  Group  C4. 


Figure  5.11. 


Survivor  functions  for  high  school  diploma  graduates  and  non-graduates 
scoring  above/below  the  mean  of  the  ABLE  composite:  Job  Group  NC3. 


Figure  5.12. 


Survivor  functions  for  high  school  diploma  graduates  and  non-graduates 
scoring  above/below  the  mean  of  the  ABLE  composite:  Job  Group  NC4. 


Discussion 


The  results  of  the  event  history  analysis  of  first-term  attrition 
indicate  that  high  school  diploma  graduate  status  remains  a  very  powerful 
predictor,  with  the  hazard  rate  for  non-graduates  being  approximately  twice 
that  for  graduates.  Nevertheless,  information  provided  by  non-cogmtive 
measures  (specifically,  scales  from  the  ABLE)  provides  significant  incremental 
prediction  to  pre-enlistment  information  currently  available  to  the  Army. 
Information  provided  by  the  AVOICE  and  the  JOB  contributed  incremental 
prediction  on  occasion,  but  both  were  overshadowed  when  the  ABLE  was  in  the 
equation. 

Fakinq/Coachinq 

Using  biodata  and  temperament  inventories  as  pre-enlistment  selection 
measures  raises  questions  of  the  possible  effects  of  coaching  and  faking  on 
applicant  responses.  Biodata  items  tend  to  be  less  susceptible  to  such 
effects  to  the  extent  that  they  are  verifiable.  Temperament  inventories, 
however,  have  been  shown  to  be  easily  faked  (e.g..  Hough,  Eaton,  Dunnette, 

Kamp  &  McCloy,  1990).  Hough  et  al.  found,  however,  that  such  distortion  does 
not  have  marked  effects  on  the  validity  of  the  instrument.  To  the  extent  that 
this  result  is  general izable,  the  concerns  about  faking  might  be  exaggerated. 
Note,  however,  that  the  respondents  in  the  Hough  et  al.  study  were  not 
coached. 

The  effects  of  coaching  on  the  validity  of  the  ABLE  was  investigated  by. 
Young,  White,  and  Oppler  (1992).  Soldiers  in  this  study  were  asked  to 
complete  the  ABLE  and  were  assigned  to  one  of  three  response  conditions: 
Honest,  Ad-Lib_Faking  (i.e.,  respondents  were  to  answer  the  questions  in  a 
mdnn6r  each  believed  would  most  impress  the  Army),  and  Coached,  Soldiers  who 
were  coached  were  given  the  same  instructions  given  to  the  Ad-Lib  Faking 
group,  but  the  respondents  were  then  given  three  practice  items  and  instructed 
which  responses  were  the  most  favorable  and  why.  Young  et  al.  examined  the 
effect  of  the  three  conditions  on  the  point-biserial  correlations  between  the 
ABLE  scales  and  a  dichotomous  criterion  variable  representing  attrition  during 
the  first  18  months  of  service  (scored  as  1  if  attrition  occurred,  0 
otherwise) . 

Young  et  al.  found  that  the  coached  respondents  scored  an  average  of  1?2 
and  0.6  standard  deviation  higher  on  the  ABLE  scales  than  did  the  Honest  and 
Ad-Lib  Faking  group  respondents.  Nevertheless,  the  correlations  across  the 
three  conditions  were  similar  in  magnitude,  with  few  significant  differences 
being  found  between  the  Honest  and  Ad-lib  Faking  conditions.  In  this  respect, 
the  results  are  similar  to  those  reported  by  Hough  et  al.  (1990)  with  regard 
to  performance  criteria.  Virtually  all  of  the  point-biserials  were  negative, 
indicating  higher  ABLE  scale  scores  were  associated  with  decreased  attrition. 
The  point-biserials  for  the  Coached  respondents,  however,  switched  sign,  all 
becoming  positive,  indicating  that  higher  ABLE  scores  were  associated  with 
increased  attrition.  Although  Young  et  al.  stated  that  the  reasons  for  this 
phenomenon  were  uncertain,  the  study  suggests  that  the  impact  of  coaching  on 
the  validity  of  the  ABLE  in  an  operational  setting  to  screen  for  first-term 
attrition  could  be  severe.  The  effects  of  coaching,  however,  might  be 
partially  mediated  by  warnings  about  scales  that  can  detect  faking,  score 
corrections  for  faking,  and  so  on. 


212 


Cross  Validity 


The  use  of  regression  typically  raises  the  question  of  cross  validity. 
Event  history  models  do  not  produce  a  measure  similar  to  the  coefficient  of 
determination,  obviating  the  use  of  shrinkage  formulae.  Nevertheless,  a 
procedure  has  been  suggested  for  assessing  the  fit  of  a  proportional  hazards 
regression  model  in  a  second  sample  (McCloy,  1993).  To  the  extent  that  the 
procedure  is  viable,  questions  of  cross  validity  may  be  addressed. 

The  issue  of  cross  validity,  however,  has  limited  application  to  first- 
term  attrition.  Attrition  depends  in  part  on  organizational  policy  that  in 
turn  depends  upon  social,  economic,  and  political  demands.  For  example,  the 
current  downsizing  of  the  military  could  result  in  the  Army  being  a  bit  more 
strict  in  enforcing  certain  standards  for  soldiers  (e.g.,  minimum  performance 
standards  and  physical  requirements)  than  at  other  times.  Optimal  equations 
for  predicting  attrition  that  are  developed  during  the  observance  of  a 
specific  organizational  policy  might  cross-val idate  well  as  long  as  the 
current  policy  remains  the  rule.  Any  change  in  policy,  however,  could  mean 
that  the  equations  are  no  longer  optimal.  A  new  analysis  would  be  necessary 
to  develop  equations  for  predicting  attrition  occurring  under  the  new 
organizational  policy. 

Component/Attrition  Relationships 

The  effects  of  two  variables  on  the  hazard  of  first-term  attrition  merit 
special  consideration.  First  is  the  positive  relationship  between  Dominance 
and  thejiazard^  indicating  that  dominant  soldiers  are  more  likely  to  exit 
prematurely.  This  relationship  might  appear  counterintuitive,  given  that 
dominance  could  be  associated  with  leadership  and  a  "take-charge"  demeanor. 
Perhaps  the  finding  supports  the  notion  that  highly  dominant  soldiers  are 
recalcitrant  and  unresponsive  to  orders,  attempting  to  take  charge  when  they 
should  follow. 

Of  similar  interest  is  the  appearance  of  the  ASVAB  Quantitative 
composite  in  three  of  the  four  best  equations.  This  composite  comprises  the 
two  mathematics  subtests.  Arithmetic  Reasoning  (AR)  and  Mathematical  Knowledge 
(MK).  These  two  subtests,  in  turn,  appear  with  the  two  verbal  subtests  of 
Word  Knowledge  (WK)  and  Paragraph  Comprehension  (PC)  in  the  Armed  Forces 
Qualification  Test,  the  military's  primary  selection  measure.  Thus,  soldiers 
have  already  been  selected  to  some  degree  on  the  Quantitative  score.  The  AFQT 
has  been  shown  time  and  again  to  have  very  little  relationship  to  attrition, 
but  the  verbal  subtests  are  given  twice  the  weight  of  the  quantitative 
subtests  when  calculating  the  present  AFQT  score.  By  removing  the 
Quantitative  subtests  from  the  AFQT  and  using  them  as  a  composite,  a  sizable 
relationship  appears  with  first-term  attrition,  notwithstanding  the 
attenuating  effects  of  range  restriction. 

Indeed,  the  lack  of  relationship  between  the  AFQT  and  first-term 
attrition  could  be  due  to  the  Quantitative  and  Verbal  subtests  working  in 
opposite  directions.  Whereas  higher  Quantitative  composite  scores  are 
associated  with  a  decreased  hazard  of  first-term  attrition,  the  parameters  for 
the  Verbal  composite  from  both  the  block  analyses  and  the  best  subset  analyses 
are  positive.  Hence,  higher  Verbal  scores  are  associated  with  an  increased 
rate  of  attrition,  with  the  relationship  being  negligible  for  the  Combat  MOS 
but  relatively  strong  for  the  Non-Combat  jobs. 
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The  relationships  of  the  Verbal  and  Quantitative  composites  might  be 
explained  in  part  by  gender  differences  in  attrition  rates  and  scores  on  the 
composites.  That  is,  men  score  higher  than  women  on  the  Quantitative 
composite  but  lower  than  women  on  the  Verbal  composite.  In  addition,  the 
attrition  rate  for  women  is  higher  than  that  for  men.  Thus,  the  Verbal  and 
Quantitative  scores  might  be  serving  as  proxy  variables  for  a  male/female 
dummy  variable.  This  would  explain  why  the  Verbal  composite  shows  no  effect 
in  the  Combat  job  groups,  where  there  are  no  females. 

To  examine  this  question,  analyses  were  conducted  for  the  Non-Combat  job 
groups  in  which  a  dummy  variable  for  gender  was  included  in  the  best  subset 
analyses.  Although  the  effect  of  the  dummy  variable  somewhat  mitigated  the 
effect  of  the  Verbal  composite,  the  composite  continued  to  appear  in  the  best 
prediction  equations  (i.e.,  the  Verbal  composite  provided  significant 
incremental  fit  of  the  proportional  hazards  regression  model).  Thus,  the 
positive  relationship  between  Verbal  scores  and  attrition  rate  in  Non-Combat 
MQS  can  not  be  entirely  attributed  to  a  gender  effect. 

Summary 

The  current  research  strongly  suggests  that  the  Army  can  improve  the 
prediction  of  first-term  attrition  by  gathering  biodata/temperament 
information  prior  to  enlistment  and  by  making  better  use  of  the  information 
they  currently  possess  (i.e.,  ASVAB  Quantitative  scores).  Regarding  future 
research,  the  hazard  functions  given  in  Figure  5.4  indicate  that  most  of  the 
attrition  occurs  in  the  first  six  months  of  service.  The  determinants  of 
early-term  attrition  (e.g.,  during  the  first  six  months)  may  differ  from  the 
determinants  of  attrition  occurring  later  in  the  term.  Such  a  hypothesis  can 
be  tested  using  event  history  analysis.  Future  research  will  be  directed  at 
this  question,  as  well  as  further  investigation  into  the  issue  of  cross 
val idity. 
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Chapter  6 

THE  ROLE  OF  JOB  SATISFACTION  IN  PERFORMANCE,  ATTRITION,  AND  REENLISTMENT 

Deirdre  J.  Knapp,  Gary  W.  Carter,  Rodney  A.  McCloy,  and  Ani  S.  DiFazio 


This  chapter  begins  with  a  background  discussion  of  the  research 
questions  that  prompted  the  inclusion  of  a  job  satisfaction  measure  in  the 
Project  A/Career  Force  research  program.  It  then  moves  to  a  description  of 
the  development  of  the  Army  Job  Satisfaction  Questionnaire  (AJSQ)  and  its 
psychometric  properties.  The  remainder  of  the  chapter  summarizes  Career  Force 
research  linking  job  satisfaction  with  job  performance  and  two  types  of 
military  turnover. 


BACKGROUND 

There  were  several  reasons  for  adding  a  job  satisfaction  instrument  to 
the  array  of  measures  administered  to  Project  A/Career  Force  soldiers.  The 
primary  purpose  of  the  measure  was  to  serve  as  a  predictor  of  turnover  --  both 
non-reenlistment  and  attrition.  The  measure  was  also  intended  to  serve  as  an 
additional  criterion  measure  that  might  be  predicted  by  measures  of  temper¬ 
ament,  interests,  and  cognitive  ability.  In  this  regard,  job  satisfaction  and 
performance  are  viewed  as  parallel  criteria.  Each  is  valued  and  the  objective 
is  to  increase  both  job  satisfaction  and  performance.  The  relationship 
between  facets  of  job  satisfaction  and  facets  of  job  performance  was  also  a 
research  issue  of  considerable  interest.  The  specific  research  questions 
which  were  addressed  with  the  Project  A/Career  Force  job  satisfaction  data  are 
reviewed  below. 


Research  Questions 


Satisfaction  and  Performance 

Research  Review.  Several  widely  cited  literature  reviews  (e.g., 
laffaldano  &  Muchinsky,  1985;  Podsakoff  &  Williams,  1986;  Vroom,  1964)  have 
presented  substantial  evidence  to  show  that  relationships  between  job 
satisfaction  and  job  performance  are  positive  but  weak.  For  example,  in  a 
meta-analysis  of  studies  of  relationships  between  job  satisfaction  and 
performance, 'laffaldano  and  Muchinsky  found  a  sample-size  weighted  mean 
correlation  between  job  satisfaction  and  job  performance  of  .146,  This  mean 
correlation  was  based  on  217  individual  correlations  (representing  a  total 
sample  size  of  12,192)  between  scales  tapping  a  number  of  different  job 
satisfaction  facets  and  many  types  of  performance  measures.  When  corrected 
for  sampling  error  and  measurement  unreliability  in  both  satisfaction  and 
performance  measures,  this  mean  correlation  rose  to  .172.  Similar  findings 
were  reported  by  Vroom  and  by  Podsakoff  and  Williams. 

Petty,  McGee,  and  Cavender  (1984)  reported  a  higher  mean  correlation 
(.23),  and  attributed  the  difference  in  mean  correlations  between  that  found 
in  their  meta-analysis  and  those  found  in  other  quantitative  reviews  to  the 
inclusion  in  their  review  of  more  studies  using  professional  and  managerial 
samples.  The  modest  size  of  the  mean  correlation  reported  by  laffaldano  and 
Muchinsky  led  these  researchers  to  suggest  that  the  satisfaction-performance 
relation  may  be  an  illusory  correlation  (see  Chapman  &  Chapman,  1969),  or  a 
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perceived  relation  between  variables  that  people  believe  should  be  related  to 
each  other,  but  in  fact  are  not. 

The  reviews  cited  above  summarized  data  across  studies  using  a  diverse 
array  of  job  performance  measures  tapping  many  different  aspects  of 
performance.  Over  the  past  decade  there  has  been  a  recognition  that  job 
satisfaction  can  reasonably  be  expected  to  be  related  to  certain  aspects  of 
performance  but  not  to  others.  With  this  has  come  a  renewed  appreciation  of 
the  importance  of  carefully  considering  the  relevance  of  predictor  and 
criterion  constructs  to  each  other  when  designing  and  interpreting  studies. 

This  recognition  led  researchers  to  examine  relationships  between  job 
satisfaction  and  extrarole,  prosocial  behavior.  Several  researchers  (e.g., 
Bateman  &  Organ,  1983;  Smith,  Organ,  &  Near,  1983)  have  found  moderate 
correlations  between  job  satisfaction  and  a  cluster  of  job  performance 
categories  that  has  been  labeled  organizational  citizenship  behavior  (OCB). 
Organ  (1988)  defined  OCB  as  "...behavior  that  is  discretionary,  not  directly 
or  explicitly  recognized  by  the  formal  reward  system,  and  that  in  the 
aggregate  promotes  the  effective  functioning  of  the  organization"  (p.  4). 

Borman  and  Motowidlo  (1993)  discussed  contextual  performance  --  job 
activities  that  are  not  part  of  the  technical  core  of  jobs,  or  of  the  "task 
performance"  aspects  of  jobs,  and  yet  contribute  in  important  ways  to 
organizational  effectiveness.  Although  OCB  is  similar  in  many  ways  to 
contextual  performance,  the  two  classes  of  behavior  do  not  completely  overlap. 
Organ  (1988)  stressed  that  OCB,  by  definition,  "is  not  an  enforceable 
requirement  of-the  role  or  the  job  description"  (p.  4)  and  is  not  "directly  or 
formally  recompensed  by  the  organization's  reward  system"  (p.  5).  Contextual 
performance  is_a  broader  construct  than  OCB,  subsuming  both  extrarole 
behaviors  not  explicitly  recognized  by  the  formal  reward  system  and  behavior 
that  is  not  part  of  the  technical  core  of  the  job  and  yet  is  explicitly 
recognized  by  the  formal  reward  system  (Borman  &  Motowidlo,  1993). 

There  are  regulations  and  standard  procedures  governing  virtually  every 
aspect  of  life  among  those  in  the  U.S.  Army.  Moreover,  mechanisms  exist 
through  which  to  formally  recognize  adherence  to  (or  lack  of  adherence  to) 
these  regulations  and  procedures.  For  this  reason,  contextual  performance 
appears  to  be  a  more  pertinent  class  of  behavior  than  OCB  to  study  in  military 
organizations. 

Hypotheses.  It  was  hypothesized  that  there  would  be  moderate 
correlations  between  job  satisfaction  and  contextual  performance  indices,  but 
negligible  correlations  between  job  satisfaction  and  scores  on  "can  do" 
measures  of  the  technical  core  of  the  job  (i.e.,  job  knowledge  and  hands-on 
test  scores).  Correlations  between  job  satisfaction  and  individual  measures 
that  appear  to  be  influenced  by  both'  contextual  performance  and  task 
performance  were  expected  to  be  lower  than  correlations  between  satisfaction 
and  more  pure  measures  of  contextual  performance,  but  higher  than  correlations 
between  satisfaction  and  "can  do"  measures  of  the  technical  core  of  the  job. 

In  addition  to  these  hypotheses  being  tested  at  the  level  of  individual 
measurement  instruments,  they  were  tested  using  the  five  factor  scores  from 
the  LVI  performance  model.  In  these  analyses,  the  Effort  and  Leadership  and 
Personal  Discipline  factor  scores  were  considered  to  be  indices  of  contextual 
performance. 
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Satisfaction  and  Turnover 


Research  Review.  A  great  deal  of  theory  development  and  research  has 
been  associated  with  the  link  between  job  attitudes  and  turnover  (see  Horn, 
Prussia,  &  Griffeth,  1992  and  Tett  &  Meyer,  1993  for  recent  reviews).  Yet  it 
is  not  clear  that  research  conducted  using  civilian  samples  is  general izable 
to  military  samples  (e.g.,  Horn,  Katerberg,  &  Hulin,  1979;  Knapp,  1993). 
Furthermore,  empirical  data  from  military  turnover  research  are  not  so 
plentiful.  Indeed,  the  available  military  research  is  rather  sparse  and  has 
been  conducted  on  diverse  samples  that  may  have  only  limited  general izabi 1 ity 
to  each  other  (e.g..  National  Guard  enlisted  versus  Regular  Army  enlisted). 

Very  little  research  has  examined  the  relationship  between  job 
satisfaction  and  military  attrition  (i.e.,  separation  before  the  enlistment 
contract  expires).  This  is  understandable  given  that  most  attrition  occurs 
early  in  the  enlistment  term  and  is  hypothesized  to  be  related  to  problems  of 
adaptability  rather  than  to  job  dissatisfaction  per  se.  Although  there  is 
more  research  examining  the  relationship  between  job  satisfaction  and  military 
reenlistment  behavior,  most  studies  of  reenlistment  focus  on  factors  other 
than  job  satisfaction  (e.g.,  pay,  reenlistment  bonuses,  spouse  support, 
education) . 

The  amount  of  relevant  military  turnover  research  is  also  reduced  by  the 
fact  that  turnover  research  data  collected  on  military  personnel  who  enlisted 
prior  to  1973  (i.e.,  before  the  end  of  the  draft)  are  considered  to  be  of 
limited  general  izabi  1-ity  to  the  All  Volunteer  Force  (Boesel  &  Johnson,  1984; 
Etheridge,  1989;  Mobley,  Hand,  Baker,  &Meglino,  1979). 

Turnover-job  satisfaction  research  conducted  on  samples  of  military 
personnel  who  enlisted  in  the  mid-1970s  or  later  includes  data  from  two 
samples  of  National  Guard  enlistees  collected  in  the  late  1970s  (Horn  et  al., 
1979;  Horn  &  Hulin,  1981;  Miller,  Katerberg,  &  Hulin,  1979).  National  Guard 
personnel  work  on  a  part-time  basis  and  their  first  enlistment  term  is  six 
years.  Satisfaction  and  reenlistment  intention  were  both  strongly  correlated 
with  reenlistment  behavior  in  both  samples. 

Three  related  studies  involved  samples  of  full-time  Navy  enlisted 
personnel.  Royle  and  Robertson  (1980)  appended  items  on  satisfaction  and 
intent  to  reenlist  onto  task  analysis  surveys  administered  to  Navy  personnel. 
Correlations  between  satisfaction  and  intent  to  reenlist  ranged  from  .19  to 
.39  in  the  four  job  samples  included  in  this  study.  Farkas  and  Tetrick  (1989) 
reported  on  a  longitudinal  study  in  which  job  satisfaction  and  reenlistment 
intention  measures  were  administered  to  sailors  at  three  points  during  their 
first-term  of  enlistment.  Satisfaction  and  intention  measured  after 
completion  of  advanced  training  were  significantly  correlated  with  each  other 
and  with  subsequent  reenlistment  behavior.  Finally,  LaRocco,  Pugh,  and 
Gunderson  (1977)  used  discriminant  analysis  to  distinguish  among  three  groups 
of  first-tour  sailors  using  several  types  of  variables,  including  satisfaction 
and  performance.  Although  the  results  suggested  that  both  satisfaction  and 
performance  contributed  to  the  prediction  of  group  membership,  the  three 
groups  used  by  these  researchers  confounded  the  criteria  of  retention  and 
attrition  (i.e.,  one  group  included  attritees  and  another  group  included 
reenl istees) . 
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Youngblood,  Mobley,  and  Meglino  (1983)  used  repeated  measures  analysis 
to  examine  satisfaction,  attrition  intention,  reenlistment  intention,  and 
turnover  (both  attrition  and  non-reenlistment)  data  from  first-term  Marine 
Corps  enlisted  personnel.  Data  were  collected  at  entry,  at  the  end  of 
training,  and  18-20  months  after  entry.  Results  of  the  analysis  confirmed 
that  the  precursors  of  turnover  showed  systematic  fluctuations  throughout  the 
enlistment  term.  Mobley  et  al.  (1979)  reported  on  an  earlier  phase  of  this 
research  which  involved  analyses  of  early  attrition  (within  the  first  11 
weeks)  only. 

Motowidlo  and  Lawton  (1984)  reported  the  only  satisfaction-turnover 
research  found  that  used  Regular  Army  enlisted  personnel.  These  researchers 
collected  ddtd  on  expectdncies,  sdtisfdction ,  dnd  reenlistment  intentions  from 
two  samples  of  soldiers  who  were  within  six  months  of  the  end  of  their  first 
enlistment  term.  Approximately  15  percent  of  the  320  soldiers  in  sample  1 
reenlisted  and  20  percent  of  the  299  soldiers  in  sample  2  reenlisted.  The 
correlation  between  satisfaction  and  reenlistment  was  .20  for  sample  1  and  .26 
for  sample  2;  correlations  between  satisfaction  and  reenlistment  intention 
were  .19  and  .32,  respectively;  and  correlations  between  reenlistment 
intention  and  reenlistment  were  .66  and  .61. 

As  indicated  by  this  review,  most  military  turnover-satisfaction 
research  has  focused  exclusively  on  reenlistment  behavior.  Job  satisfaction 
and  intention  to  reenlist  have  consistently  been  found  to  be  related  to 
reenlistment  behavior.  Many  questions  remain,  however.  Only  one  of  these 
studies  included  a  measure  of  performance  (LaRocco  et  al.,  1977).  Others  have 
not  reported  indices  of  association  (or  model  fit)  for  the  job  satisfaction 
and  reenlistment  intention  measures  as  predictors  of  turnover.  Clearly,  there 
is  more  to  be  ^earned  about  affective  precursors  to  turnover  in  the  military 
arena,  even  at  the  most  elementary,  bivariate  levels  of  analysis. 

HvDOtheses.  It  was  hypothesized  that  both  job  satisfaction  and 
reenlistment  intention  would  be  significantly  correlated  with  reenlistment. 

It  wds  further  hypothesized  thdt  job  sdtisfdction  would  provide  incrementdl 
predictive  value  for  predicting  reenlistment  over  that  provided  by  intention 
to  reenlist.  Parallel  hypotheses  were  formulated  for  the  prediction  of 
attrition.  That  is,  it  was  expected  that  job  satisfaction  would  be 
significantly  related  to  attrition  even  when  the  intention  to  attrit  was 
included  in  the  prediction  equation.  Although  it  was  expected  that  the  models 
would  fit  better  when  the  event  to  be  predicted  was  restricted  to  voluntary 
attrition,  this  hypothesis  could  not  be  tested  because  model  fit  was 
confounded  with  attrition  base  rate.  That  is,  restricting  the  event  of 
interest  to  voluntary  attrition  reduced  the  base  rate  which,  in  turn,  led  to 
decrements  in  model  fit. 

Performance  and  Turnover 

Research  Review.  McEvoy  and  Cascio  (1987)  conducted  a  meta-analysis  of 
the  relationship  between  performance  and  turnover.  The  mean  correlation, 
weighted  by  sample  size,  was  -.28.  An  appreciable  amount  of  unexplained 
variance  was  still  apparent  after  corrections  were  made  for  sampling  error  and 
measurement  unreliability.  Moreover,  only  two  of  the  24  correlations  included 
in  their  meta-analysis  were  from  military  samples.  Specifically,  two  of  the 
correlations  came  from  the  study  of  Navy  regular  enlisted  personnel  by  LaRocco 
et  al.  (1977)  described  earlier. 
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Hypotheses.  Because  of  the  paucity  of  directly  related  research  job 
performance  indices  were  included  in  the  reenlistment  and  attrition  analyses 
on  an  exploratory  basis.  That  is,  no  specific  hypotheses  related  to  job 
performance  were  proposed. 


DEVELOPMENT  OF  THE  ARMY  JOB  SATISFACTION  QUESTIONNAIRE  (AJSQ) 

Given  the  types  of  research  questions  described  above,  the  project 
needed  an  instrument  with  which  to  measure  job  satisfaction.  It  was  not 
feasible  to  use  a  standardized  instrument  (e.g.,  the  Job  Descriptive  Index, 
from  Smith,  Kendall,  &  Hulin  (1969))  for  this  purpose  because  of  copyright 
issues.  Moreover,  although  the  Army  has  had  a  longstanding  interest  in 
measuring  aspects  of  satisfaction,  motivation,  and  morale  (Borman  &  Bleda, 
1978;  Motowidlo  et  al.,  1976),  a  review  of  the  instruments  used  in  the 
military  literature  (e.g..  Army  Early  Career  Satisfaction  Survey,  Army 
Experience  Survey,  Commander's  Unit  Analysis  Profile,  Cureton's  Airman  Morale 
Survey)  did  not  identify  instruments  that  suited  the  needs  of  the  project. 
These  existing  instruments  were,  however,  used  to  help  construct  the  Army  Job 
Satisfaction  Questionnaire  (AJSQ).  The  AJSQ  was  developed  to  provide  reliable 
and  valid  indices  of  satisfaction  with  important  aspects  of  Army  jobs  and 
overall  job  satisfaction.  Moreover,  it  was  designed  to  be  easy  to  understand 
and  quick  to  administer. 

Initial  Development  and  Field  Testing 

A-pool  of  several  hundred  items  was  assembled  from  previously  developed 
military  instruments  related  to  satisfaction.  Five  project  researchers 
independently  reviewed  the  items  and  identified  several  common  facets  of 
satisfaction  that  might  be  useful  to  incorporate  in  the  new  instrument.  A 
draft  of  the  AJSQ  was  developed  by  drawing  items  from  existing  instruments  to 
cover  each  of  the  categories  of  satisfaction  selected  for  measurement.  A 
second  section  of  the  draft  AJSQ  was  designed  to  help  explain  differences  in 
job  satisfaction  levels  expressed  in  Section  I.  Items  in  Section  II  were 
related  primarily  to  reasons  for  enlistment  and  reenlistment  intentions,  and 
most  were  drawn  from  other  Army  surveys  (e.g.,  the  Army's  New  Recruit  Survey). 


The  first  draft  of  the  AJSQ  was  reviewed  by  several  project  researchers 
and  then  revised  for  field  testing.  Section  I  of  the  field  test  version 
comprised  a  total  of  62  items  covering  six  facets  of  satisfaction: 
satisfaction  with  supervision,  co-workers,  promotions,  pay,  work,  and  the  Army 
as  an  organization.  Section  II  was  also  revised  and  expanded  to  contain  30 
items. 


The  field  test  version  of  the  AJSQ  and  the  short  form  of  the  Minnesota 
Satisfaction  Questionnaire  (MSQ)  (Weiss,  Dawis,  England,  &  Lofquist,  1967) 
were  administered  to  271  first-  and  second-tour  soldiers  at  three  Army 
installations.  The  MSQ  is  a  highly  regarded  20-item  measure  of  job 
satisfaction.  It  was  used,  with  permission  from  its  authors,  to  serve  as  a 
marker  measure  in  this  research;  that  is,  the  construct  validity  of  the  AJSQ 
was  evaluated  by  examining  its  relationship  to  scores  on  the  MSQ. 

Factor  analysis  of  the  Section  I  satisfaction  items  resulted  in  a 
relatively  clean  representation  of  the  expected  six-factor  structure.  The 
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total  number  of  items  was  reduced  from  62  to  37  by  eliminating  items  which 
exhibited  relatively  low  loadings  on  their  respective  factors.  Scale  scores 
were  constructed  by  summing  the  constituent  items  within  each  factor. 
Correlations  between  these  scales  and  related  scales  from  the  MSQ  are  shown  in 
Table  6.1. 

The  pattern  of  correlations  demonstrates  reasonable  levels  of 
discriminant  and  convergent  validity  for  the  AJSQ  subscales.  The  only 
exception  to  this  finding  is  that  the  MSQ  scale  on  satisfaction  with  company 
policies  and  practices  was  more  highly  correlated  with  the  AJSQ  work  scale 
than  with  the  Army  scale.  The  AJSQ  Army  scale,  however,  is  considerably 
broader  in  concept  than  the  MSQ  Policy  and  Practices  scale,  so  a  particularly 
high  correlation  would  not  have  been  expected.  It  was  simply  the  scale  which 
corresponded  most  closely  conceptually  with  the  Army  scale  for  satisfaction 
with  the  Army. 


Table  6.1 

Correlations  of  Army  Job  Satisfaction  Questionnaire  (AJSQ)  Scales  With  Key 
Minnesota  Satisfaction  Questionnaire  (MSQ)  Scales _ 


AJSQ  Scale 


MSQ  Sccrie 

.Supervisors 

Co- 

Workers 

Promotions 

Pay 

Work 

Army 

Supervision  — 

.59 

.30 

.23 

.10 

.38 

.30 

Human  Relations 

Supervision  — 

.52 

.20 

.19 

.36 

.30 

.25 

Technical 

Co-Workers 

.10 

.43 

.13 

.19 

.21 

.15 

Advancement 

.33 

.26 

.56 

.25 

.41 

.37 

Compensation 

.28 

.20 

.27 

.47 

.33 

.37 

Variety 

.29 

.25 

.21 

.10 

.52 

.18 

Ab i 1 i ty 

Uti 1 ization 

.40 

.34 

.33 

.18 

.62 

.32 

Achievement 

.34 

.34 

.31 

.23 

.59 

.30 

Company  Pol icies/ 

.29 

.23 

.30 

.31 

.48 

.41 

Practices 

Note.  N  =  271.  Correlations  between  similar  scales  are  underlined. 
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Final  Contents  of  the  AJSQ 


To  conserve  administration  time,  the  AJSQ  was  shortened  again  before  it 
was  administered  to  soldiers  in  the  LVI/CVII  data  collection.  Section  I  of 
the  LVI/CVII  version  contained  20  items  related  to  the  six  facets  of 
satisfaction  with  three  to  four  items  defining  each  facet.  Section  II 
included  13  items. 

Prior  to  administration  of  the  questionnaire  to  soldiers  in  LVII,  the 
AJSQ  was  revised  once  again.  In  Section  I,  the  satisfaction  assessment 
portion,  two  items  were  added  to  the  work  subscale  and  a  single  overall 
satisfaction  item  was  added  as  well.  Whereas  the  other  satisfaction  items 
used  a  5-point  scale  ranging  from  very  dissatisfied  to  very  satisfied,  the 
overall  item  used  a  7-point  adjectival  scale  developed  by  Ironson  and  Smith 
(1981). 

Section  II  was  substantially  revised  and  expanded.  Four  items  were 
dropped  and  25  items  were  added  for  a  total  of  34  items.  Again,  most  of  the 
items  came  from  existing  Army  surveys  (e.g..  Army  Career  Satisfaction  Survey, 
Army  Families  Research  Questionnaire).  They  cover  (a)  reasons  for  enlisting, 
(b)  reenlistment  intentions,  (c)  unit  morale,  (d)  perceived  fairness  of 
treatment,  and  (e)  the  impact  of  socio-political  events  (e.g..  Operation 
Desert  Storm,  Army  downsizing)  that  had  occurred  since  the  LVI  AJSQ  measure 
was  administered. 

A  copy  of  the  LVII  version  of  the  AJSQ  is  included  as  Appendix  C.  It  is 
annotated  to  show  the  items  that  did  not  appear  on  the  LVI/CVII  version. 

Psychometric  Properties 

AJSQ  data  were  collected  from  11,140  soldiers  in  the  first-tour 
Longitudinal  Validation  sample  (LVI),  1,025  second-tour  soldiers  from  the 
Concurrent  Validation  sample  (CVII),  and  1,574  second-tour  soldiers  from  the 
Longitudinal  Validation  sample  (LVII). 

As  previously  described,  there  were  some  differences  in  the  two  versions 
of  the  AJSQ  used  to  collect  these  data;  in  particular,  the  satisfaction 
assessment  section  of  the  LVII  form  had  three  items  that  did  not  appear  on  the 
LVI/CV.II  form.  The  factor  structure  and  scoring  rules  used  for  the  two 
versions  are  described  in  this  section.  Descriptive  statistics  are  provided 
for  the  two  Longitudinal  Validation  samples  (i.e.,  LVI  and  LVII).  This 
discussion  is  restricted  to  the  satisfaction  items  in  Section  I  of  the  AJSQ. 

Prior  to  finalizing  the  AJSQ  scoring  scheme,  a  principal  components 
analysis  was  conducted  on  the  20  satisfaction  items  comprising  the  LVI/CVII 
version  of  the  instrument.  The  results  showed  a  very  clean  representation  of 
the  expected  six-factor  structure.  Table  6.2  shows  the  same  analysis,  with 
similar  results,  conducted  using  the  23  items  from  the  LVII  version  of  the 
instrument.  This  table  reflects  a  varimax  (orthogonal)  rotation;  results 
using  an  oblique  rotation  were  highly  similar.  Note  that  overall  satisfaction 
item  loads  most  highly  (.61)  on  the  satisfaction  with  work  factor  but  also  has 
a  reasonably  high  loading  ^40)  on  the  satisfaction  with  the  Army  factor. 

Based  on  the  factor  structure  of  the  instrument,  it  seemed  reasonable  to 
construct  scale  scores  for  each  of  the  six  factors  and  to  consider  all  items 
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Table  6.2 


AJSQ  LVII  Principal  Components  Analysis:  Rotated  Factor  Pattern  Matrix 


Satisfaction  With: 


Item^ 

Supervisors 

Co-Workers 

Promotions 

Pay 

Work 

Army 

Item  01 

.90 

.05 

.05 

.03 

.14 

.08 

.84 

Item  02 

.90 

.07 

.03 

.03 

.14 

.08 

.84 

Item  03 

.87 

.05 

.08 

.05 

.21 

.10 

.82 

Item  04 

.06 

.74 

.08 

.04 

.14 

-.01 

.58 

Item  05 

.06 

.83 

.05 

.03 

.15 

.06 

.73 

Item  06 

.02 

.81 

.04 

.09 

.15 

.16 

.71 

Item  07 

.05 

.04 

.90 

.10 

.10 

.02 

.83 

Item  08 

.07 

.09 

.86 

.10 

.13 

.21 

.82 

Item  09 

.05 

.07 

.91 

.09 

.13 

.15 

.88 

Item  10 

.02 

.06 

.11 

.89 

.09 

.22 

.87 

Item  11 

.05 

.06 

.10 

.87 

.04 

.16 

.81 

Item  12 

.04 

.06 

.08 

.89 

.08 

.23 

.86 

Item  13 

.08 

.10 

.02 

.04 

.83 

.12 

.72 

Item  14 

.09 

.10 

.05 

.03 

.87 

.07 

.79 

Item  15- 

.09 

.11 

.10 

.06 

.85 

.05 

.  76 

Item  16 

.08 

.11 

.09 

.03 

.88 

.10 

.80 

Item  17  _ 

.10 

.08 

.10 

.07 

.85 

.05 

.75 

Item  18 

.15 

.10 

.07 

.06 

.79 

.18 

.70 

Item  19 

.09 

.08 

.19 

.16 

.12 

.65 

.52 

Item  20 

.06 

.06 

.04 

.13 

.10 

.81 

.  69 

Item  21 

.05 

.01 

.03 

.17 

.11 

.81 

.71 

Item  22 

.06 

.07 

.14 

.16 

.19 

.80 

.73 

Overal 1 

.18 

.13 

.13 

.07 

.61 

.40 

.61 

Satisfaction 
(Item  23) 

E igenvalue 

2.50 

2.02 

2.53 

2.51 

4.95 

2.87 

17.38 

Note,  =  communal ity  (sum  of  squared  factor  loadings)  for  variables. 
Loadings  expected  to  be  high  on  each  factor  are  shown  in  bold. 

®  See  Appendix  C  for  item  content. 
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suitable  for  scoring.  The  scores  were  constructed  by  computing  the  mean  of 
the  constituent  items  within  each  factor.  Because  the  LVII  version  of  the 
instrument  had  additional  items,  two  sets  of  work  subscores  were  computed  for 
the  LVII  sample;  one  includes  the  two  new  items  and  the  other  does  not. 

An  overall  satisfaction  composite  was  constructed  by  calculating  the 
mean  across  all  items  appearing  in  Section  I  of  the  LVI/CVII  version  of  the 
AJSQ.  Again,  two  versions  of  the  overall  composite  score  were  calculated  for 
the  LVII  sample,  one  that  includes  the  two  additional  items  on  the  LVII  form 
and  one  that  does  not.  The  LVII  form  also  includes  the  single  overall  job 
satisfaction  item,  which  was  scored  separately. 

Table  6.3  lists  all  the  scores  that  were  generated  from  the  AJSQ  and 
their  respective  internal  consistency  reliability  estimates.  Table  6.4  shows 
the  means  and  standard  deviations  for  AJSQ  scores.  As  would  ordinarily  be 
expected,  satisfaction  levels  for  second-tour  soldiers  tended  to  be  higher 
than  those  for  first-tour  soldiers.  Finally,  scale  intercorrelations  are 
provided  in  Table  6.5.  Again,  as  expected,  scales  show  low  to  moderate 
correlations  with  each  other. 


Table  6.3 


AJSQ  Scoring  System  and 

Internal  Consistency  Reliability 

Estimates 

Scale 

Constituent  Items^ 

(LVII  Numbering) 

Coefficient  Alpha 
LVI  LVII 

Supervision  “ 

1,  2,  3 

.90 

.90 

Co-Workers 

4,  5,  6 

.80 

.75 

Promotions 

7,  8,  9 

.91 

.90 

Pay 

10,  11,  12 

.91 

.90 

Work 

14,  15,  17,  18 

.90 

.90 

Work  1 1 

13,  14,  15,  16,  17,  18 
(LVII  sample  only) 

-- 

.94 

Army 

19,  20,  21,  22 

.84 

.83 

Overall  Composite 

Items  1-22,  excluding  13  &  16 

.90 

.87 

Overall  Composite  II 

Items  1-22  (LVII  sample  only) 

— 

.89 

Overall  Satisfaction 

23 

-- 

-- 

®  See  Appendix  C  for  item  content. 
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Table  6.4 


AJSO  Descriptive  Statistics 

LVI 

LVII 

Scale 

Mean  SD 

Mean 

SD 

Supervision 

3.15  1.09 

3.47 

1.05 

Co-Workers 

3.29  .88 

3.56 

.78 

Promotions 

2.57  1.12 

2.75 

1.11 

Pay 

2.61  1.02 

2.64 

1.02 

Work 

2.79  1.06 

3.27 

1.09 

Work  II 

-- 

3.30 

1.07 

Army 

2.51  .90 

2.84 

.86 

Overall  Composite 

2.80  .67 

3.08 

.60 

Overall  Composite  II 

-- 

3.11 

.62 

Overall  Satisfaction^ 

__ 

4.45 

1.21 

Note.  LVI  n  =  11,140;  LVII  n 

=  1,574. 

^  Note  that  this  item  is  rated  on  a  7-point  scale, 

whereas  the  other 

means  are 

based  on  5-point  scales. 

Table  6.5 

AJSQ  Score  Intercorrelations 


Scale 

Supervisors 

Co- 

Workers 

Promotions  Pay 

Work 

Army 

Comp 

Supervision 

.19 

.30 

.21 

.38 

.33 

.62 

Co-Workers 

.16 

-- 

.25 

.17 

.31 

.28 

.52 

Promotions 

.16 

.18 

— 

.33 

.38 

.39 

.67 

Pay 

.13 

.17 

.25 

— 

.30 

.48 

.62 

Work 

.32 

.31 

.25 

.18 

— 

.42 

.74 

Army 

.22 

.20 

.30 

.42 

.30 

-- 

.75 

Composite 

.55 

.50 

'  .59 

.57 

.70 

.68 

-- 

Overall  Satisfaction 

.32 

.28 

.28 

.24 

.63 

.47 

.64 

Note.  Upper  diagonal 

is  LVI  (n 

=  11,140); 

lower 

diagonal 

is  LVII 

(n  = 

1,574). 
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ANALYSES,  RESULTS,  AND  DISCUSSION 


The  analyses  that  were  conducted  and  the  results  that  were  found  are 
presented  and  discussed  in  turn  for  (a)  the  relationship  of  satisfaction  to 
performance  and  (b)  the  prediction  of  turnover. 

In  this  report,  the  research  questions  associated  with  satisfaction, 
performance,  and  turnover  were  addressed  using  data  from  the  LVI  sample. 
Related  analyses  using  the  CVII  and  LVII  data  will  be  described  in  a  later 
report.  Note  also  that  the  analyses  reported  in  this  chapter  are  restricted 
to  the  nine  Batch  A  MOS  because  most  required  soldiers  to  have  a  complete 
array  of  job  performance  data. 

Satisfaction  and  Performance 


Sample  and  Measures 

The  sample  for  this  set  of  analyses  comprised  6,352  LVI  soldiers  from 
the  nine  Batch  A  MOS.  In  addition  to  the  six  subscores  and  AJSQ  composite 
score  described  above,  a  "Challenge"  subscore  was  computed  by  summing  two 
items  tapping  satisfaction  with  job  challenge  and  satisfaction  with  the 
opportunity  to  use  skills  and  abilities. 

Several  indices  of  contextual  and  non-contextual  performance  were  used 
in  the  analyses  reported  here.  (See  Chapter  1  for  a  complete  listing  of  the 
criterion  measures  used  in  each  sample  and  the  scores  that  were  derived  from 
them.)  ^he  following  scores  came  from  the  Army-Wide  rating  scales:  an 
overall  composite  created  by  averaging  across  all  dimensions,  the  overall 
effectiveness  rating,  the  NCO  potential  rating,  and  three  composites  formed  by 
summing  across”dimensions  (i.e.,  Technical  Skill  and  Effort,  Personal 
Discipline,  and  Physical  Fitness  and  Military  Bearing).  The  Personal 
Discipline  ratings  composite  can  be  considered  a  measure  of  contextual 
performance. 

Another  contextual  performance  composite  was  created  by  summing  scores 
on  three  dimensions  tapping  effort,  integrity,  and  self-development;  this 
composite  is  referred  to  as  the  Contextual  Army-Wide  composite.  A  Contextual 
Combat  Prediction  Scales  composite  was  constructed  by  summing  scores  across 
10  of  the  19  items  on  the  Combat  Scales.  Three  administrative  scores  from  the 
Personnel  File  Form  were  used  (i.e..  Awards  and  Certificates,  Disciplinary 
Actions,  and  M16  Qualification).  Overall  total  scores  were  derived  for  the 
hands-on  tests,  job  knowledge  tests,  and  MOS-specific  rating  scales.  Finally, 
the  five  factor  scores  from  the  LVI  performance  model  were  used;  two  of  these 
scores  (i.e.,  Effort/Leadership  and  Personal  Discipline)  may  be  considered 
indices  of  contextual  performance. 

Results 


Table  6.6  shows  sample-size  weighted  mean  correlations  between  the  eight 
job  satisfaction  scores  and  19  job  performance  scores  across  the  nine  MOS. 
Correlations  based  on  the  measures  of  contextual  performance  (i.e.,  Contextual 
Army-wide  composite.  Contextual  Combat  Scales  composite.  Personal  Discipline 
Army-wide  composite,  the  Personal  Discipline  factor  from  the  performance 
model,  and  the  Effort  and  Leadership  factor  from  the  performance  model)  are 
printed  in  bold  type.  Both  peer  and  supervisor  performance  rating  data  were 
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Note:  Sample  sizes  range  from  5,222  to  6,345.  Performance  rating  scores  are  based  on  combined  peer  and  supemsor  rating  data. 
Correlations  based  on  measures  of  contextual  performance  are  printed  in  bold  type. 


used  in  computing  the  correlations  shown.  Mean  correlations  across  MOS  are 
presented  because  analyses  showed  that,  except  for  correlations  based  on 
satisfaction  with  promotions,  most  or  all  of  the  inter-occupation  variance  in 
these  correlations  could  be  attributed  to  sampling  error.  This  finding  is 
probably  due  to  differential  promotion  rates  across  MOS. 

Several  points  should  be  noted  from  Table  6.6.  First,  the  correlations 
tend  to  be  low. 

Second,  the  relative  magnitudes  of  the  correlations  based  on  different 
performance  indices  support  the  a  priori  hypotheses.  The  correlations  between 
job  satisfaction  and  contextual  performance  measures  are  higher  than  the 
correlations  between  job  satisfaction  and  other  performance  indices. 
Correlations  between  job  satisfaction  and  measures  that  are  probably 
influenced  by  both  contextual  performance  and  task  performance  (e.g.,  MOS- 
Specific  rating  scale  score,  Disciplinary  Actions,  Physical  Fitness/Military 
Bearing  performance  model  factor  score)  are,  in  general,  lower  than 
correlations  between  job  satisfaction  and  more  pure  measures  of  contextual 
performance,  but  are  higher  than  correlations  between  satisfaction  and  "can 
do"  measures  of  the  technical  core  of  the  job. 

Correlations  based  on  "can  do"  measures  of  the  technical  core  of  the  job 
(e.g.,  hands-on,  job  knowledge,  and  M16  Qualification  scores)  tend  to  be 
negligible.  The  difference  in  the  correlations  would  most  likely  be  even 
greater  if  the  differential  reliabilities  of  the  performance  measures  were 
taken  into  account.  The  "can  do"  factors  are  not  based  on  the  rating  method 
and  are  somewhat  more  reliable  than  the  "will  do"  factors. 

The  only  correlations  that  do  not  support  the  hypotheses  are  those  based 
on  the  Awards  and  Certificates  variable.  Scores  on  this  administrative 
measure  should~be  influenced  by  both  contextual  performance  and  task 
performance  and  yet  correlations  between  job  satisfaction  and  scores  on  this 
variable  are  negligible. 

Third,  patterns  of  correlations  vary  widely  by  job  satisfaction 
dimension.  Examining  the  correlations  based  on  the  Personal  Discipline 
performance  model  scores  provides  a  good  feel  for  the  relative  magnitude  of 
correlations  with  various  satisfaction  scales.  The  highest  correlation  (.237) 
is  based  on  the  overall  composite  job  satisfaction  scale.  In  decreasing  order 
of  magnitude  the  other  seven  correlations  are:  promotions  (.233);  supervision 
(.174);  work  (.158);  Army  (.146);  challenge  (.134);  pay  (.124);  and  co-workers 
(.088).  The  relative  order  of  the  satisfaction  scales  in  terms  of  the 
magnitudes  of  their  correlations  with  other  performance  dimensions  is  similar 
to  this  for  most  performance  dimensions. 

Sample-size  weighted  mean  correlations  between  job  satisfaction  and 
performance  based  on  supervisor  evaluations  of  performance  are  shown  in  the 
top  half  of  Table  6.7.  Similar  correlations  based  on  peer  evaluations  of 
performance  are  shown  in  the  bottom  half  of  the  same  table.  A  comparison  of 
the  correlations  based  on  supervisor  and  peer  ratings  reveals  consistently 
higher  correlations  between  job  satisfaction  and  supervisor  evaluations  of 
performance  than  between  job  satisfaction  and  peer  evaluations  of  performance, 
except  for  the  correlations  based  on  the  pay  and  co-workers  subscales. 
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It  is  possible  that  correlations  between  satisfaction  and  supervisor 
performance  evaluations  are  hipher  than  correlations  between  satisfaction  and 
peer  performance  evaluations  because  supervisor  ratings  are  more  reliable.  To 
explore  this  possibility,  the  correlations  shown  in  Table  6.7  based  on  the 
three  Army-wide  rating  composites,  the  overall  effectiveness  ratings,  and  the 
NCO  potential  ratings  were  corrected  for  criterion  unreliability.  (Interrater 
reliability  estimates  used  in  computing  the  corrected  correlations  ranged  from 
.56  to  .64  for  the  supervisor  ratings  and  .48  to  .59  for  the  peer  ratings.) 
Correcting  for  criterion  unreliability  did  not  substantially  influence  the 
relative  sizes  of  the  correlations.  Correlations  based  on  supervisor  data  were 
still  consistently  higher  than  correlations  based  on  peer  data,  with  the 
exception  of  correlations  based  on  the  co-workers  satisfaction  scale. 

Corrected  correlations  between  satisfaction  with  co-workers  and  peer 
evaluations  of  performance  were  higher  than  corrected  correlations  between 
satisfaction  with  co-workers  and  supervisor  evaluations  of  performance. 

Summary 


The  hypotheses  regarding  satisfaction  and  performance  were  supported  by 
the  pattern  of  correlations  found.  Sample-size  weighted  mean  correlations 
between  job  satisfaction  scores  and  scores  on  measures  of  contextual  job 
performance  were  higher  than  correlations  between  job  satisfaction  and  scores 
on  other  job  performance  measures.  Sample-size  weighted  mean  correlations 
between  job  satisfaction  and  measures  influenced  by  both  contextual  performance 
and  task  performance  were  lower  than  correlations  between  job  satisfaction  and 
more  pure  measures  of  contextual  performance.  Correlations  based  on  "can  do" 
task  performance  measures  were,  in  most  cases,  negligible. 

ATthough~the  expected  pattern  of  correlations  was  found,  the  correlations 
based  on  measures  of  contextual  performance  were  lower  than  expected,  and  lower 
than  those  found  in  some  of  the  previous  studies  of  relationships  between  job 
satisfaction  and  OCB  (e.g.,  Bateman  &  Organ,  1983;  Smith  et  al.,  1983).  The 
very  reason  that  led  to  the  use  of  measures  of  contextual  performance  instead 
of  measures  of  OCB  in  this  study  may  explain  the  lower  than  expected 
correlations.  The  fact  that  regulations  and  standard  procedures  govern  most 
aspects  of  Army  life  and  that  mechanisms  exist  to  formally  recognize  adherence 
to  those  regulations  and  procedures  makes  it  more  likely  that  a  person  will 
perform  prosocial  acts  --  in  part,  to  avoid  organizationally  sanctioned 
punishments  for  failing  to  perform  them,  or  to  reap  organizationally  sanctioned 
benefits  for  performing  them.  Thus,  performance  differences  resulting  from 
differences  in  affective  states  may  be  more  subtle  in  military  contexts  than  in 
organizational  contexts  in  which  there  are  fewer  strictures  governing  behavior. 

For  all  satisfaction  scales  except  the  pay  and  co-workers  scales, 
correlations  between  satisfaction  and  supervisor  performance  ratings  were 
higher  than  correlations  between  satisfaction  and  peer  performance  ratings. 
These  differences  cannot  be  attributed  to  higher  inter-rater  reliabilities 
among  the  supervisor  rating  data  than  among  the  peer  rating  data.  It  may  be 
that  supervisors  are  simply  more  likely  than  peers  to  allow  the  attitudes  of 
those  they  are  rating  to  influence  their  performance  judgments,  regardless  of 
whether  these  attitudes  are  reflected  in  actual  performance  differences. 
However,  a  more  likely  explanation  is  that  supervisors  are  better  able  to 
discern  subtle  performance  differences  that  covary  with  differences  in 
satisfaction. 
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Prediction  of  Turnover 


Sample  and  Measures 

The  base  sample  for  the  analyses  reported  here  comprises  5,721  LVI 
soldiers  representing  the  nine  Batch  A  MOS.  Most  analyses  used  the  overall 
satisfaction  composite  score  and  an  overall  job  performance  score.  The  overall 
performance  score  was  computed  by  summing  the  five  standardized  factor  scores. 

An  intention-to-reenl ist  variable  was  extracted  from  Section  II  of  the 
AJSQ.  It  was  scored  as  follows:  (0)  intend  to  leave  the  Army,  (1)  do  not 

know,  and  (2)  intend  to  reenlist.  Approximately  69  percent  of  the  soldiers  in 
this  sample  (Batch  A  MOS;  all  enlistment  terms)  indicated  that  they  would  leave 
the  Army,  15.5  percent  indicated  that  they  would  reenlist,  and  15.5  percent 
indicated  that  they  were  undecided. 

An  intention-to-attrit  variable  was  obtained  from  a  survey  designed  by 
the  Army  Family  Research  Program  which  was  administered  in  conjunction  with  the 
LVI  data  collection  (Research  Triangle  Institute,  1988).  It  was  scored  (0)  if 
the  soldier  indicated  that  he  or  she  was  likely  to  leave  the  Army  before  the 
enlistment  term  expired  and  (1)  if  the  soldier  expected  to  complete  his  or  her 
tour.  About  4  percent  of  the  sample  reported  that  they  intended  to  attrit. 

It  was  expected  that  soldiers  having  more  time  remaining  in  their  first 
term  would  be  more  likely  to  attrit,  regardless  of  their  level  of  job 
satisfaction  or  job  performance.  The  fact  that  3-year  enlistees  had  a  10 
percent  attrition  rate  during  the  period  between  testing  and  the  end  of  their 
enlistment  term,  compared  to  a  20.5  percent  attrition  rate  for  4-year 
enlistees,  supports  this  hypothesis.  Base  rate  differences  across- enl istment 
terms  were  controlled  by  conducting  turnover  analyses  for  soldiers  in  each 
enlistment  term  separately. 

A  special  variable  was  constructed  to  provide  a  control  variable  for 
differences  in  time  remaining  for  soldiers  within  each  enlistment  term.  This 
variable  is  expressed  as  the  proportion  of  the  soldier's  term  that  had  expired 
at  the  time  that  the  research  measures  were  administered.  The  mean  proportion 
of  term  expired  was  .72  for  2-year  term  soldiers,  .55  for  3-year  term  soldiers, 
and  .36  for  4-year  te'f'm  soldiers.  As  expected,  the  mean  proportion  of  term 
that  had  expired  at  the  time  of  testing  decreased  as  the  length  of  the 
enlistment  term  increased. 

Turnover.  For  most  of  the  reenlistment  analyses,  reenlistees  were 
compared  to  soldiers  who  completed  their  initial  term  of  enlistment  and  who 
were  eligible  to  reenlist.  Soldiers  who  left  the  Army  before  completing  their 
enlistment  terms  or  who  were  otherwise  ineligible  to  reenlist  were  excluded 
from  these  analyses.  Excluding  soldiers  who  were  ineligible  to  reenlist  was 
generally  appropriate  because  two  of  the  three  primary  predictors  (intention 
and  satisfaction)  were  hypothesized  to  be  related  to  voluntary  rather  than 
involuntary  turnover  behavior.  Note,  however,  that  analyses  which  use  job 
performance  as  the  only  type  of  predictor  include  soldiers  who  were  ineligible 
for  reenlistment.  The  reenlistment  rate  across  MOS  and  enlistment  terms, 
excluding  those  soldiers  who  were  ineligible,  was  38  percent.  Adding  in  the 
soldiers  who  were  ineligible  for  reenl istmen-t,  the  rate  dropped  to  32  percent. 
This  is  substantially  higher  than  the  reenlistment  intention  rate  which  was  15 
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percent,  a  finding  that  is  consistent  with  prior  military  research  (Hiller, 
1982). 


For  the  attrition  analyses,  soldiers  who  separated  prematurely  for 
avoidable  reasons  were  compared  to  those  who  did  not.  When  event  history 
analysis  was  used  to  conduct  the  data  analysis,  no  observations  had  to  be 
excluded  from  the  analysis.  When  logistic  regression  analysis  was  used, 
unavoidable  attrition  cases  (e.g.,  attrition  due  to  disability)  were  dropped. 
Note  that  the  percentage  of  soldiers  who  exited  prematurely  is  small  in  this 
sample  (13%)  because  attrition  is  most  likely  to  occur  early  in  the  first 
enlistment  term.  The  timing  of  the  LVI  data  collection  was  such  that  soldiers 
had  to  have  already  made  it  well  into  their  first  term  in  order  to  be  included 
in  this  analysis  sample. 

Bivariate  correlations  among  the  variables  described  above  are  presented 
in  Table  6.8.  As  hypothesized,  reenlistment  intention  and  job  satisfaction 
were  significantly  correlated  with  reenlistment.  Job  satisfaction  and 
attrition  intention  were  also  significantly  correlated  with  attrition.  Job 
performance  was  signicantly  correlated  with  both  types  of  turnover-related 
behavior  as  well. 


Table  6.8 


Intercorrelations 

Among  Major  Turnover  Analysis  Variables 

JS 

JP 

Reeni ist 
Intent 

Attrit 

Intent 

Reeni ist 

Attrit 

Job  Satisfaction 

1.000 

Job  Performance 

.189 

1.000 

Reenlist  Intention 

1  .268 

.073 

1.000 

Attrit  Intention^ 

.196 

.192 

.099 

1.000 

Reeni istment^ 

.158 

.132 

.369 

.104 

1.000 

Attrition'^ 

-.089 

-.263 

-.060 

-.264 

-.263 

1.000 

Proportion  Term 
Completed 

-.081 

.126 

-.024 

.067 

.026 

-.196 

Note,  n  =  4,098  for  all  correlations  involving  attrition  intention  and  5,721 
for  all  others.  All  correlations  except  those  in  italics  are 
significant  at  p  =  .0001. 


^  Given  the  96%  intention-not-to-attrit  base  rate,  the  maximum  possible  rp^  is 
approximately  .49;  attrition  coded  1  if  soldier  plans  to  complete  term  and  0 
if  not. 

Given  the  32%  reenlistment  base  rate,  the  maximum  possible  rp,,  is  about  .78; 
reenlistment  coded  1  if  reenlisted  and  0  if  not. 

Given  the  13%  attrition  base  rate,  the  maximum  possible  value  rpj,  is 
approximately  .60;  attrition  coded  1  if  attrited  and  0  if  not. 
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Prediction  of  Reenlistment 


Satisfaction.  Logistic  regression  with  stepwise  selection  was  calculated 
in  an  effort  to  identify  the  satisfaction  subscales  most  strongly  predictive  of 
reenlistment.  However,  the  results  were  difficult  to  interpret  because 
multicollinearity  (moderate  to  high  intercorrelations)  among  the  satisfaction 
subscales  resulted  in  sign  reversals  in  some  of  the  parameter  estimates;  for 
example,  pay  satisfaction  appeared  to  be  negatively  related  to  reenlistment 
when  in  fact  it  is  positively  related.  In  view  of  the  problems  involved  in 
subscale  analyses,  the  overall  job  satisfaction  composite  was  used  in 
subsequent  analyses  dealing  with  predictions  of  turnover. 

Performance.  The  relationship  between  job  performance  and  reenlistment, 
exclusive  of  other  predictors,  was  examined  in  two  stages  of  analysis.  First, 
the  six  indices  of  performance  that  are  administrative  in  nature  (i.e..  Awards 
and  Certificates,  Disciplinary  Actions,  Skill  Qualification  Test  scores,  M16 
Qualification,  Physical  Readiness  test  score,  and  Promotion  Rate)  were 
regressed  onto  reenlistment.  These  stepwise  logistic  regression  analyses  were 
conducted  separately  for  each  term  of  enlistment,  and  soldiers  ineligible  for 
reenlistment  were  included.  The  administrative  measures  were  looked  at 
separately  from  the  for-research-only  measures  (i.e.,  hands-on  tests,  written 
job  knowledge  tests,  and  peer/supervisor  ratings)  because  soldiers  have 
knowledge  of  their  level  of  performance  on  these  measures  which  may,  in  turn, 
influence  their  reenlistment  decision. 

In  the  second  set  of  analyses,  the  administrative  measures  demonstrating 
a  significant  relationship  to  reenlistment  were  combined  with  overall  scores 
from  the  for-research-only  measures  to  predict  reenlistment.  It  was  expected 
that  the* administrative  measures  would  be  more  useful  than  the  for-research- 
only  measures  for  predicting  reenlistment  behavior. 

With  regard  to  the  first  set  of  analyses,  none  of  the  administrative 
measures  met  the  significance  criterion  for  entering  the  prediction  equation  in 
either  set  of  analyses  for  the  soldiers  with  2-year  enlistment  terms. 
Disciplinary  Actions  and  Promotion  Rate  entered  the  equations  for  both  3-year 
and  4-year  enlistment  term  soldiers.  In  addition,  the  Awards  and  Certificates 
score  was  a  significant  predictor  of  4-year  term  soldier  reenlistment.  The  M16 
Qualification  score  met  the  entry  criterion  for  3-year  term  soldiers,  but  it 
had  such  a  weak,  negative  beta  weight  that  it  was  not  used  in  the  second  set  of 
analyses . 

The  results  of  the  second  set  of  analyses  are  shown  in  Table  6.9.  For 
both  3-year  and  4-year  term  soldiers,  the  overall  performance  ratings  composite 
(constructed  using  Army-wide  and  MOS-specific  ratings  dimensions)  was  the  only 
for-research-only  measure  that  entered  the  prediction  equation.  This  makes 
sense  given  that  supervisor  evaluations  help  determine  reenlistment 
eligibility.  Using  the  correlation  between  the  probability  of  reenlistment 
estimated  by  the  regression  equation  and  actual  reenlistment  as  a  gauge,  the 
relationship  between  the  individual  performance  measures  and  reenlistment  is 
relatively  small,  with  a  point-biserial  correlation  of  -.169  for  3-year  term 
soldiers  and  -.183  for  4-year  term  soldiers. 
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Table  6.9 

Prediction  of  Reenlistment  Using  Individual  Job  Performance  Measures 


Predictor 


Parameter  Standard  Odds 

Estimate  Error _ p-val  ue _ 


3-Year  Term  (n  =  3,613) 


Intercept 

-4.983 

.463 

.0001 

.007 

Awards  and  Certificates 

— 

— 

—  —  — 

Disciplinary  Actions^ 

.206 

.051 

.0001 

1.229 

Promotion  Rate 

.038 

.008 

.0001 

1.039 

Ratings 

.004 

.001 

.0001 

1.004 

Tpb  =  .169^ 

36%  reenl  istment"^ 

4-Year  Term  (n  =  1,912) 

Intercept 

-5.479 

.718 

.0001 

.004 

Awards  and  Certificates 

.069 

.016 

.0001 

1.072 

Disciplinary  Actions® 

.221 

.081 

.0061 

1.248 

Promotion  Rate 

.034 

.012 

.0036 

1 . 035 

Ratings 

.004 

.001 

.0048 

1.004 

■■pb  “  •  iss" 

29%  reenl  istment'^ 


a 

b 


c 


Score  has  been  reversed-coded  so  relationship  to  reenlistment  is  expected  to 

be  positive.  ^  ..  u  -i  -  x  ^ 

r  i s  the  point-biserial  correlation  between  predicted  probability  of 

reenlistment  and  actual  reenlistment. 

Including  soldiers  ineligible  for  reenlistment. 


Overall  Analyses.  The  relative  contribution  of  the  three  types  of 
predictors  (i.e.,  intention,  job  satisfaction,  and  job  performance)  was 
assessed  by  evaluating  the  improvement  of  model  fit  with  the  addition  of  each 
predictor  to  the  prediction  eguation.  Intention  was  the  first  predictor 
entered  into  the  model  because  it  is  generally  believed  to  be  the  most 
immediate  precursor  to  turnover  (e.g.,  Mobley  et  al. ,  1979).  The  overall  job 
satisfaction  composite  was  entered  next.  Overall  job  performance  was  entered 
last  because  it  was  not  expected  to  be  related  as  strongly  to  reenlistment  as 
satisfaction  would  be.  Separate  reenlistment  analyses  were  conducted  for 
soldiers  in  each  enlistment  term  (2,  3,  and  4  years). 

The  effect  of  adding  each  variable  to  the  model  was  evaluated  by 
conducting  likelihood  ratio  tests.  A  likelihood  ratio  test  is  conducted  by 
computing  the  difference  between  the  logs  of  the  likelihood  ratios  for  each 
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model  (i.e.  the  model  with  the  new  variable  and  the  model  without  the  new 
variable).  This  difference,  multiplied  by  -2,  is  distributed  asymptotically  as 
chi-square,  with  degrees  of  freedom  in  this  case  equal  to  one.  In  addition,  an 
index  of  the  predictive  strength  of  each  model  was  constructed  by  calculating 
the  point-biserial  correlation  between  the  predicted  probability  of 
reenlistment,  as  determined  by  each  model,  and  actual  reenlistment  behavior. 

Results  of  the  comparison  among  reenlistment  prediction  models  are 
summarized  in  Table  6.10.  For  all  three  enlistment  terms,  all  three  models  are 
statistically  significant.  However,  the  incremental  contribution  of  job 
satisfaction  and  job  performance  over  reenlistment  intention  appears  to  be 
minimal.  There  are  no  significant  differences  among  the  three  models  for  2- 
year  enlistees.  The  addition  of  job  satisfaction  provides  incremental 
predictive  power  over  intention  for  3-year  enlistees,  but  the  addition  of  job 
performance  to  those  two  predictors  does  not  improve  the  model  fit.  In 
contrast  the  addition  of  job  satisfaction  to  the  model  does  not  significantly 
improve  model  fit  for  4-year  enlistees,  but  the  addition  of  job  performance 
does.  Clearly,  the  most  powerful  and  consistent  predictor  of  reenlistment 
behavior  in  this  sample  is  reenlistment  intention.  This  conclusion  is 
supported  by  examining  the  prediction  equation  parameters  provided  in  Table 

6.11. 


Table  6.10 

Point-Biserial  Correlations  Between  Reenlistment  and  Predicted  Probability  of 
Reenlistment  for  Three  Models _ _ _ 


Enlistment  Term 


Alternative  Models 

2-Year 

3-Year 

4-Year 

Intention 

.548 

.293 

+  Satisfaction 

.553  (.546)^ 

.402  (.400)J 

.293  (.287)')^ 

+  Performance 

.553  (.544) 

.402  (.399) 

.301  (.293)3 

Reenlistment  Rate*^ 

14% 

45% 

39% 

Maximum  rpi^ 

.60 

.81 

.80 

Note.  All  models  have  chi-squares  significant  at  p.  <  .05.  2-year  n  -  492; 
3-year  n  =  2,753;  4-year  n  =  1,322. 

^  Correlations  in  parentheses  are  adjusted  for  shrinkage  using  formula  from 
Stein  (1960). 

^  Differences  in  model,  chi-squares  are  significant  at  £  <  .05. 

Excluding  soldiers  ineligible  for  reenlistment. 

Estimated  from  Figure  4-5  in  Nunnally  (1967,  p.  133). 
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Table  6.11 

Prediction  of  Reeni istment  Using  Three  Post-Enlistment  Predictors 


Predictor 


Parameter  Standard 

Estimate  Error 


£ 


Odds 

Ratio 


2-Year  Term  (n  =  492) 


Intercept 

-3.889 

1.714 

.0232 

.020 

Intention 

1.990 

.236 

.0001 

7.319 

Satisfaction 

.395 

.266 

.1375 

1.485 

Performance 

.001 

.003 

.8256 

1.001 

14%  reeni istment^ 

;-Year  Term  (n  =  2,753) 

Intercept 

-1.494 

.470 

.0015 

.224 

Intention 

1.068 

.059 

.0001 

2.911 

Satisfaction 

.157 

.066 

.0169 

1.170 

Performance 

.000 

.000 

.5479 

1.001 

45%  reenlistment® 

i-Year  Term  (n  =  1,322) 

Intercept 

-2.571 

.681 

.0002 

.076 

Intention  _ 

.779 

.080 

.0001 

2.179 

Satisfaction 

.015 

.094 

.8730 

1.015 

Performance 

.003 

.001 

.0118 

.084 

39%  reeni istment^ 


^  Excluding  soldiers  ineligible  for  reenlistment. 


The  point-biserial  correlations  between  the  predicted  probability  of 
reenlistment  and  actual  reenlistment  suggest  that  the  reenlistment  behavior  of 
3-year  enlistees  is  predicted  more  successfully  than  that  of  the  4-year 
enlistees.  Despite  a  relatively  low  reenlistment  base  rate,  the  reenlistment 
behavior  of  2-year  enlistees  is  most  accurately  predicted.  This  finding  is 
particularly  interesting  given  that  the  maximum  point-biserial  correlation 
possible  with  the  2-year  term's  14  percent  reenlistment  base  rate  (rpt,=  .60)  is 
considerably  lower  than  that  possible  with  the  3-year  term's  45  percent  base 
rate  (rh=.81).  It  is  reasonable  to  expect,  however,  that  the  more  distant 
the  solaier  is  from  making  a  reenlistment  decision  at  the  time  of  testing,  the 
less  accurate  the  prediction  is  going  to  be.  Moreover,  some  2-year  term 
enlistees  may  have  already  made  their  reenlistment  decisions  prior  to  the  time 
of  testing. 
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Two  sets  of  analyses  were  used  to  model  attrition.  The  first  set  used 
logistic  regression  analysis  to  assess  the  incremental  contribution  of  four 
variables  to  the  prediction  of  avoidable,  late-term  attrition.  These 
variables  were  (a)  the  proportion  of  term  completed  when  measures  were 
administered,  (b)  intention  to  attrit,  (c)  overall  job  satisfaction,  and  (d) 
overall  job  performance. 

In  the  second  set  of  analyses,  event  history  analysis  was  used  to 
determine  whether  job  satisfaction  contributed  significantly  to  model  fit  over 
the  seven  best  predictors  identified  by  the  analyses  reported  in  Chapter  5  of 
this  report  (i.e.,  high  school  diploma  status,  ASVAB  Quantitative  composite, 
and  the  following  scores  from  the  ABLE:  Nondel iquency,  Dominance,  Physical 
Condition,  Self-Esteem,  and  Social  Desirability). 

In  both  sets  of  analyses,  the  event  of  interest  was  avoidable  attrition 
(both  voluntary  and  involuntary)  as  described  in  Knapp  (1993).  All  other 
turnover-related  outcomes  (i.e.,  reenlistment,  completion  of  first  term, 
unavoidable  attrition)  were  treated  as  censored  observations  in  the  event 
history  analyses.  In  the  logistic  regression  analyses,  unavoidable  attritions 
were  dropped  from  the  analysis  sample. 

Analyses  were  performed  separately  for  soldiers  with  3-year  and  4-year 
enlistment  terms.  Due  to  small  sample  sizes,  data  from  soldiers  with  2-year 
terms  of  enlistment  were  not  analyzed.  As  expected,  the  base  rate  of 
attrition  was  markedly  higher  for  4-year  enlistees  (20.5%)  than  for  3-year 
enlistees  (10%).  For  the  event  history  analyses,  analyses  were  also  conducted 
separately  for  combat  and  non-combat  MOS  and  were  stratified  by  MOS.  This  is 
the  same  analysis  approach  described  in  Chapter.  5  except  that  only  soldiers  in 
Batch  A  MOS  were  included. 

Intention.  Satisfaction,  and  Performance.  The  first  set  of  analyses 
were  intended  to  assess  the  incremental  contribution  of  four  variables  to  the 
prediction  of  avoidable  attrition.  This  was  done  by  comparing  the  overall  fit 
of  the  following  four  nested  models:  (a)  proportion  of  term  completed  when 
measures  were  administered,  (b)  proportion  of  term  completed  and  intention  to 
attrit,  (c)  proportion  of  term  completed,  intention,  and  satisfaction,  and 
(d)  proportion  of  term  completed,  intention,  satisfaction,  and  performance. 

The  proportion  of  term  completed  when  the  intention,  satisfaction,  and 
performance  measures  were  administered  was  considered  a  statistical  control 
variable  and  was  therefore  always  entered  first  into  the  model.  The  other 
variables  were  entered  into  the  model  in  the  order  of  the  expected  strength  of 
their  relationship  to  attrition  (i.e.,  intention,  satisfaction,  performance). 

The  four  models  are  compared  in  Table  6.12.  All  models  were 
statistically  significant  (£  <  .05)  for  both  3-  and  4-year  enlistees. 
Likelihood  ratio  tests  of  the  model  chi-squares  indicate  that  the  addition  of 
each  predictor  provided  incremental  predictive  power  for  4-year  term 
enlistees,  and  for  all  predictors  except  job  satisfaction  for  the  3-year  term 
enlistees.  There  was  little  difference  between  enlistment  terms  in  the 
predictive  power  of  the  full  model.  The  correlations  between  the  predicted 
probability  of  attrition  and  attrition  were  -.372  (3-year  term)  and  -.370 


Table  6.12 


Point-Biserial  Correlations  Between  Attrition  and  Predicted  Probability  of 
Attrition  for  Four  Mode  1 _ _ _ _ _ 


Alternative  Models 


_ Enlistment  Term 

3-Year  _ 4-Year 


Proportion  of  term  completed 

-.115 

+  Intention 

-.300 

+  Satisfaction 

-.302 

+  Performance 

-.370 

Avoidable  attrition  rate 

Maximum 

1 

(-.297)  ) 

-.144 

-.258 

(-.250) 

(-.298)  ) 

-.281 

(-.271) 

r 

(-.366)  J 

'-.372 

(-.363) 

10% 

20.5% 

.55 

.70 

Note.  All  models  have  chi-squares  significant  at  £  <  .05.  3-year  n  -  2,540; 
4-year  n  =  1,197. 

^  Correlations  in  parentheses  are  adjusted  for  shrinkage  using  formula  from 
Stein  (1960). 

^  Differences  in  model  chi-squares  are  significant  at  £  <  .05. 

^  Estimated  from  Figure  4-5  in  Nunnally  (1967,  p.  133). 


(4-yedr  term).  Parameter  estimates  for  the  model  including  all  four 
predictors  are  provided  in  Table  6.13. 

Pre-Enlistment  Predictors  and  Satisfaction.  Results  of  the  event 
history  analyses  examining  pre-enlistment  predictors  and  job  satisfaction  as 
predictors  of  attrition  are  provided  in  Tables  6.14  through  6.17.  Whether  job 
satisfaction  provides  incremental  model  fit  over  the  pre-enlistment  predictors 
was  determined  using  likelihood  ratio  tests.  In  all  cases  (3-year  term, 
combat  MOS;  4-year  term,  combat  MOS;  3-year  term,  non-combat  MOS;  and  4-year 
term,  non-combat  MOS),  satisfaction  significantly  improved  model  fit. 

Examination  of  the  equation  parameters  reveals  that,  unlike  the  analyses 
presented  in  Chapter  5,  most  of  the  pre-enlistment  predictors  were  not 
significantly  or  consistently  related  to  attrition.  Whereas  all  seven  of 
these  predictors  were  related  to  attrition  in  earlier  analyses,  the  ABLE 
Nondelinquency  composite  and  job  satisfaction  were  the  only  predictors 
consistently  shown  to  be  significantly  related  to  attrition  in  these  analyses. 
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Table  6.13 

Prediction  of  Attrition  Using  Post-Enlistment  Predictors _ 

Parameter  Standard  Odds 

Predictor  Estimate  Error  Ratic 


3-Year  Term  (n  =  2,540) 


Intercept  -8.220 
Prop,  term  completed  2.796 
Intention  2.022 
Satisfaction  -003 
Performance  *015 


10%  avoidable  attrition 
4-Year  Term  (n  =  1,197) 


Intercept  -7.888 
Prop,  term  completed  3.021 
Intention  1.251 
Satisfaction  .278 
Performance  -013 


20.5%  avoidable  attrition 


.760 

.0001 

.000 

.527 

.0001 

16.375 

.261 

.0001 

7.557 

.109 

.9765 

1.003 

.001 

.0001 

1.015 

.842 

.0001 

.000 

.820 

.0002 

2.509 

.281 

.0001 

3.494 

.120 

.0202 

1.320 

.002 

.0001 

1.013 
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Table  6.14 


Prediction  of  Attrition  Using  Pre-Enlistment  Predictors:  Combat,  3-Year  Job 
Group  (C3)^ _ _ _ _ _ _ _ 


Parameter 

Standard 

Odds 

Predictor 

Estimate 

Error 

£ 

Ratio 

HSDG 

-.471 

.202 

.0197 

0.624 

QUANTITATIVE  Composite 

-.010 

.006 

.1252 

0.990 

Physical  Condition  Scale 

.010 

.031 

.  7577 

1.010 

Dominance  Scale 

.018 

.023 

.4239 

1.019 

Self-Esteem  Scale 

-.012 

.029 

.6757 

0.988 

Nondelinquency  Scale 

-.037 

.016 

.0252 

0 . 964 

Social  Desirability  Scale 

.019 

.025 

.4457 

1.020 

Satisfaction 

-.410 

.115 

.0004 

0.663 

-2  Log  L 

1  (df=l) 

Pre-enlistment  Predictors 

1946.17 

+  Satisfaction 

1933.57 

12.6* 

*  Significant  at  £  <  .01. 

^  n  =  1,_609.  Overall  avoidable  attrition  rate  is  11%. 
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Table  6.15 


Prediction  of  Attrition  Using  Pre-Enlistment  Predictors:  Combat,  4-Year  Job 


broup 

Parameter 

Standard 

Odds 

Predictor 

Estimate 

Error 

£ 

Ratio 

HSDG 

-.199 

.319 

.5329 

0.819 

QUANTITATIVE  Composite 

-.004 

.009 

.5981 

0.995 

Physical  Condition  Scale 

.079 

.043 

.0630 

1.083 

Dominance  Scale 

.077 

.031 

.0142 

1.080 

Self-Esteem  Scale 

-.023 

.038 

.5489 

0.977 

Nondelinquency  Scale 

-.109 

.022 

.0001 

0.896 

Social  Desirability  Scale 

.032 

.038 

.  3966 

1 .032 

Satisfaction 

-.563 

.173 

.0011 

0.570 

-2  Log  L 

(df=l) 

Pre-enlistment  Predictors 

892.62 

+  Satisfaction 

881.72 

10.9* 

*  Significant  at  £  <  .01. 

^  n  =  SZh.  Overall  avoidable  attrition  rate  is  17%. 
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Table  6. 16 


Prediction  of  Attrition  Using  Pre-Enlistment  Predictors:  Non-Combat,  3-Year 
Job  Group  (NC3)^ _ _ _ _ _ _ _ 


Predictor 

Parameter 

Estimate 

Standard 

Error 

£ 

Odds 

Ratio 

HSDG 

-.567 

.208 

.0065 

0.567 

QUANTITATIVE  Composite 

-.020 

.006 

.0008 

0.980 

Physical  Condition  Scale 

-.002 

.026 

•  9469 

0.998 

Dominance  Scale 

.044 

.021 

.0372 

1 .045 

Self-Esteem  Scale 

-.043 

.025 

.0918 

0.958 

Nondelinquency  Scale 

•  -.032 

.015 

.0313 

0.969 

Social  Desirability  Scale 

.050 

.023 

.0281 

1 .051 

Satisfaction 

-.460 

.106 

.0001 

0.631 

-2  Log  L  X 

^  (df=l) 

Pre-enlistment  Predictors 

2204.17 

+  Satisfaction 

2185.30 

18.87* 

*  Significant  at  £  <  .01. 

^  n  =  1^36.  Overall  avoidable  attrition  rate  is  11%. 
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Table  6.17 


Prediction  of  Attrition  Using 
Job  Group  (NC4)^ 

Pre-Enlistment  Predictors:  Non- 

-Combat,  4-Year 

Parameter  Standard 

Odds 

Predictor 

Estimate  Error 

2 

Ratio 

HSDG 

-.314  .254 

.2168 

0.731 

QUANTITATIVE  Composite 

-.006  .007 

.4320 

0.994 

Physical  Condition  Scale 

-.049  .031 

.  1088 

0.952 

Dominance  Scale 

.021  .022 

.3570 

1.021 

Self-Esteem  Scale 

.003  .030 

.9247 

1.003 

Nondelinquency  Scale 

-.044  .017 

.0098 

0.956 

Social  Desirability  Scale 

.037  .028 

.1870 

1 . 038 

Satisfaction 

-.325  .125 

.0093 

0.723 

-2  Log  L 

{df=l) 

Pre-enlistment  Predictors 

1542.07 

+  Satisfaction 

1535.40 

6.67* 

*  Significant  at  £  <  .01. 

^  n  =  674.  Overall  avoidable  attrition  rate  is  22%. 
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Th6  diffcrGDCB  bstwecri  thG  two  sets  of  analyses  is  that  those  reported 
in  Chapter  5  pertain  to  attrition  occurring  throughout  the  first  enlistment 
term,  and  the  analyses  reported  here  relate  only  to  attrition  that  occurs 
relatively  late  in  the  enlistment  term.  These  findings  support  the  notion 
that  late-term  attrition  and  the  more  common  early-term  attrition  are  not 
equivalent  criteria.  They  also  show  that,  despite  this  apparent 
nonequivalence,  the  Nondelinquency  measure  retains  substantial  predictive 
power  throughout  the  first  term  of  enlistment. 


SUMMARY 

The  AJSQ  is  a  reliable  and  valid  measure  of  satisfaction  and  it  has 
added  another  dimension  to  the  Project  A/Career  Force  research  program. 
Satisfaction  is  related  to  contextual  measures  of  performance,  though  not  as 
strongly  as  anticipated.  (Both  satisfaction  and  performance  are  related  to 
reenlistment,  but  much  of  their  predictive  power  is  eroded  when  reenlistment 
intention  is  added  into  the  prediction  equation.)  The  findings  regarding 
attrition  are  similar  to  those  associated  with  reenlistment,  although 
performance  retains  appreciable  predictive  power  even  when  intention  to  attrit 
is  included.  Satisfaction  does  predict  attrition  very  well  compared  to  pre¬ 
enlistment  cognitive  ability  and  temperament  predictors. 

A  final  observation  is  that  the  length  of  the  enlistment  term  and  the 
point  in  that  term  at  which  attitudes  and  performance  are  measured  have  a 
substantial  impact  on  the  ability  to  predict  turnover-related  behavior. 

This  chapter  has  focused  on  relationships  between  job  satisfaction,  job 
perform^ce,  and  two  types  of  military  turnover-  (reenlistment  and  attrition) 
in  the  first  tour.  Using  Project  A/Career  Force  data,  an  extensive  analysis 
of  job  satisfaetion  as  a  criterion  measure  to  be  predicted  has  been  reported 
by  Carter  (1991).  Additional  analyses  in  the  future  will  be  concerned  with 
the  consistency  of  individual  differences  in  job  satisfaction  over  time  and 
with  the  influence  of  different  frames  of  reference  on  individual  job 
satisfaction. 
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Chapter  7 

OVERALL  SUMMARY  AMD  FUTURE  PLANS 
John  P.  Campbell 
SUMMARY  OF  YEAR  FOUR 


During  the  fourth  year  of  the  Career  Force  Project,  two  major  sets  of 
analyses  related  to  predicting  second-tour  performance  were  carried  out.  The 
first  was  the  basic  validation  of  the  ASVAB  and  the  Experimental  Battery 
against  the  Longitudinal  Sample  second-tour  performance  measures  (LVII).  Much 
larger  sample  sizes  were  available  for  LVII  than  for  the  earlier  analyses  of 
the  evil  sample  and  the  predictor  information  was  much  more  complete.  The 
second  set  of  LVII  analyses  examined  how  accurately  second-tour  performance 
could  be  predicted  from  first-tour  performance  and  also  from  performance 
during  training.  The  similarity  in  the  latent  structure  of  performance  at 
each  of  the  three  levels  (i.e.,  training,  first  tour,  second  tour)  permits  the 
individual  performance  factors  to  be  examined  in  terms  of  their  divergent  and 
convergent  relationship  across  levels. 

In  addition  to  the  analyses  of  the  LVH  data,  the  LVI  data  base  was  used 
for  three  major  purposes.  First,  an  extensive  analysis  of  the  absolute  and 
discriminant  validity  for  optimal  combinations  of  the  ASVAB  subtests  plus  the 
Project  A  ECAT  battery  was  carried  out,  using  software  that  compared  all 
possible  combinations  of  tests  at  each  stage.  Second,  the  full  Longitudinal 
Validation  predictor  sample  was  used  to  examine  the  relationship  of  the 
Experimental  Predictor  Battery  to  first-term  attrition,  using  event  history 
(survival)  analysis.  Finally,  the  extensive  Project  A/Career  Force  data  base 
generated  by  the  Army  Job  Satisfaction  Questionnaire  (AJSQ)  was  used  to 
estimate  the  relationship  of  job  satisfaction  to  the  components  of  first-tour 
performance,  second-tour  performance,  first-term  attrition,  and  reenl istment/ 
non-reenl istment. 


LVII  Basic  Validation 

A  complete  LVII  validation  required  that  a  reasonable  subset  of  the 
soldiers  in  the  original  LV  predictor  sample  (N  =  28,000  for  the  nine  Batch  A 
MOS)  finish  their  first  tour  of  duty  successfully,  reenlist  for  a  second  tour, 
and  be  available  at  a  designated  set  of  data  collection  sites  for  performance 
assessment  during  the  spring  and  summer  of  1992  (i.e.,  approximately  the  third 
year  of  their  second  tour).  Sufficient  data  on  both  the  predictor  battery  and 
the  criterion  measures  were  available  for  seven  of  the  nine  Batch  A  MOS,  and 
the  total  sample  size  was  approximately  1,200. 

Multiple  correlations  were  computed  for  each  predictor  set  (ASVAB, 
spatial,  computer,  ABLE,  AVOICE,  JOB)  against  each  of  13  criterion  scores  (the 
six  LVII  performance  factors,  two  modifications  of  the  Leadership  factor 
score,  a  "will  do"  composite,  a  "can  do"  composite,  the  total  of  the  six 
factor  scores,  and  separate  scores  for  the  Hands-On  and  Job  Knowledge  tests). 

In  general,  the  validity  of  the  ASVAB  is  as  high  for  predicting  second- 
tour  performance  as  it  is  for  predicting  first-tour  performance.  Its  highest 
validities  are  for  predicting  Core  Technical  Performance,  but  it  also  has  high 
validity  for  predicting  second-tour  leadership  performance  (average  R  =  .50), 
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even  when  the  Leadership  criterion  score  is  purged  of  all  possible  method 
variance. 

The  Personal  Discipline  and  the  Fitness  and  Bearing  factors  are  not 
predicted  as  well  in  second  tour  as  they  were  in  first  tour  by  any  of  the 
tests  in  the  predictor  battery.  Possible  explanations  are  the  restriction  in 
range  on  these  criterion  variables  (unrelated  to  ASVAB)  for  the  more  highly 
selected  second-tour  samples  and  the  change  in  the  meaning  of  the  indices  for 
more  highly  selected  and  experienced  soldiers. 

The  validity  of  the  ABLE  (administered  at  the  time  of  accession)  for 
predicting  the  Leadership  factor  in  LVII  was  about  the  same  as  the  ABLE 
validity  for  predicting  the  more  heterogeneous  ELS  factor  in  CVI.  Finally, 
the  incremental  validity  of  the  Experimental  Battery  over  ASVAB  was  not  as 
great  in  the  LVII  sample  as  it  was  in  CVI  and  LVI. 

Prediction  of  Future  Performance  From  Past  Performance 

The  convergent  and  divergent  validity  of  specific  components  of  current 
performance  for  predicting  future  performance  was  examined  by  computing  three 
intercorrelation  matrices.  Respectively,  the  three  matrices  were  (a)  the 
intercorrelations  of  training  performance  criterion  measures  with  first-tour 
performance  measures,  (b)  the  intercorrelations  of  first-tour  performance 
measures  with  second-tour  performance  measures,  and  (c)  the  intercorrelations 
of  training  performance  measures  with  second-tour  performance  measures  (i.e., 
six  years  later).  The  variables  in  each  matrix  included  the  simple  sum  factor 
scores  from  each  performance  model,  the  "can  do,"  "will  do,"  and  total  factor 
composite  scores,  and  a  number  of  scoring  variations  that  attempted  to  equate 
the  content  of_the  factor  scores  precisely  across  cohorts.  Also  of  interest 
was  the  validity  with  which  early-stage  peer  and  supervisory  assessment  of  NCO 
potential  can  predict  subsequent  NCO  performance  during  the  second  tour. 

The  results  exhibited  a  striking  degree  of  convergent  and  divergent 
validity  across  the  three  organizational  levels.  In  addition,  the  magnitude 
of  the  correlations  of  performance  with  performance  was  quite  high. 

Performance  in  training  does  predict  performance  on  the  job  (first  tour),  and 
entry-level  performance  does  predict  second-tour  NCO  performance,  and  with 
considerable  differential  validity  across  performance  components.  There  are 
also  substantial  correlations  between  performance  in  training  (AIT)  and 
performance  as  an  NCO  six  years  later. 

Finally,  a  series  of  stepwise  analyses  of  the  predictor  battery  versus 
current  performance  as  predictors  of  future  performance  were  carried  out.  The 
results  clearly  showed  that  the  test  battery  and  measures  of  current 
performance  each  contribute  considerable  unique  variance  to  the  prediction  of 
future  performance. 

In  general,  the  cross-level  analyses  of  the  Project  A/Career  Force 
performance  measures  provide  considerable  additional  evidence  for  their 
construct  validity. 


ECAT  Optimal  Battery  Analysis 

The  Enhanced  Computer  Administered  Test  Battery  (ECAT)  analyses  were 
conducted  using  Project  A/Career  Force  data,  to  assist  the  Manpower  Accession 
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Policy  Working  Group  in  its  deliberations  about  possible  revisions  to  the 
ASVAB.  The  overall  purpose  was  to  examine  seven  tests  from  the  Project 
A/Career  Force  Experimental  Battery  in  terms  of  their  potential  usefulness  as 
additions  to,  or  replacements  for,  the  ten  subtests  of  the  current  ASVAB  The 
seven  candidate  tests  were: 

1)  Spatial:  Assembling  Objects 

2)  Spatial:  Orientation 

3)  Spatial:  Reasoning 

4)  Short-Term  Memory  (computerized) 

5)  Target  Identification  (computerized) 

6)  One-Hand  Tracking  (computerized) 

7)  Two-Hand  Tracking  (computerized) 

^  ,,  procedure  was  to  first  specify  that  the  Arithmetic  Reasoning 

and  Word  Knowledge  subtests  of  the  ASVAB  would  be  the  base  components  of  every 
potential  battery.  Then  all  possible  combinations  of  a  specific  number  of  the 
remaining  15  tests  (8  ASVAB  and  7  ECAT)  would  be  evaluated  on  several  indices. 
For  example,  if  9  tests  were  to  be  added,  all  possible  combinations  of  9  tests 
from  the  pool  of  15  were  evaluated.  The  number  of  additional  tests  to  be 
considered  was  determined  by  the  desired  battery  length  (in  terms  of  battery 
administration  time).  Three  different  battery  lengths  were  evaluated  (74-104 
minutes,  134-164  minutes,  194-224  minutes);  the  longest  interval  corresponds 
to  the  current  ASVAB  administration  time. 

The  indices  computed  for  each  unique  test  battery  were  as  listed  below 
All  of  the  indices  are  based  on  the  data  generated  by  computing  multiple 
correlations  of  the  battery  with  a  particular  criterion  in  each  of  the  Batch  A 
MOS.  That  IS,  the  regression  weights  are  computed  within  MOS  each  time.  For 
each  battery  there  are  nine  such  estimates.  Each  set  of  MOS  specific  weights 
tor  the  battery  was  then  applied  to  the  data  in  each  of  the  other  Batch  A  MOS 
to  compute  a  generalized  validity  estimate.  There  are  9  x  8,  or  72,  such 
estimates.  The  indices  are  defined  as  follows: 

•  Mean  absolute  validity  is  the  average  of  the  nine  MOS  specific 
estimates  after  they  have  been  corrected  by  the  Rozeboom  formula. 

•  Mgan  general izable  validity  is  the  average  of  the  72  off-diagonal 
values  obtained  when  weights  computed  on  one  MOS  are  applied  to 
the  data  from  the  other  eight  MOS  to  compute  a  correlation  between 
the  criteria  and  a  weighted  predictor  composite. 

•  Discriminant  validity  is  the  difference  between  mean  absolute 
validity  and  mean  general izable  validity  for  a  specific  test 
battery. 

•  Lhe  Brogden  Index  of  Classification  Efficiency  (BCE)  is  computed 
as  R V 1  -  5,  when  R  is  the  mean  absolute  validity  across  jobs  and 
r  is  the  mean  intercorrelation  among  the  predicted  criterion 
scores  for  each  job.  If  the  selection  ratio  and  the  number  of 
jobs  remain  constant,  then  BCE  is  isomorphic  with,  but  not  in  the 
same  metric  as  the  gain  in  mean  predicted  performance  (MPP)  that 
results  from  classification. 


247 


•  Subqroup  diffprsnce  is  th6  difference  in  mean  predictor  composite 
scores  for  Whites/Blacks,  Whites/Hispanics ,  and  Men/Women, 
computed  on  the  combined  sample  for  each  of  the  nine  sets  of  MOS- 
specific  weights  and  then  averaged  over  the  nine  estimates  of  the 
differences. 

The  entire  set  of  analyses  was  carried  out  for  each  of  three  criteria: 
the  TECH  score  from  the  End-of-Training  (EOT)  School  Knowledge  test;  the  Core 
Technical  Proficiency  (CTP)  factor  score  for  first-tour  performance;  and  the 
Hands-On  test  score.  The  discriminant  validity  index  and  the  Brogden 
Classification  Efficiency  Index  were  clearly  related. 

There  was  a  positive  correlation  among  all  the  battery  indices.  That 
is  the  greater  the  absolute  validity  of  a  battery,  the  greater  the 
discriminant  validity  and  the  greater  the  subgroup  differences.  In  general, 
the  White/Black  differences  were  more  highly  correlated  with  absolute  validity 
than  with  discriminant  validity,  and  also  classification  efficiency,  but  the 
reverse  was  true  for  Male/Female  (differences. 

The  effects  of  increasing  battery  length  were  relatively  small. 

Absolute  validity  increased  the  most  between  the  short  and  medium  length 
batteries  while  discriminant  validity  increased  slightly  but  uniformly  as  a 
function  of  increasing  test  length. 

The  ASVAB  subtests  made  their  greatest  contribution  to  absolute  validity 
rather  than  discriminant  validity  and  the  ECAT  tests  tended  to  make  their 
greatest  contribution  to  discriminant  validity.  When  maximizing  discriminant 
validity,  the  ASVAB  subtests  that  were  replaced  most  often  by  ECAT  tests  were 
General  Science,  Mechanical  Knowledge,  and  Numerical  Operations.  The  ECAT 
tests  most  often  substituted  for  ASVAB  subtests  were  Assembling  Objects, 

Target  Identification,  and  One-Hand  Tracking. 

The  most  general  conclusion  that  can  be  drawn  from  the  data  is  that  all 
the  relevant  indices  cannot  be  maximized  by  the  same  test  battery.  Although 
there  is  a  relatively  high  correlation  between  absolute  validity  and 
discriminant  validity,  it  is  not  unity  and  maximizing  discriminant  validity 
will  detract  somewhat  from  absolute  validity.  Choosing  a  battery  to  reduce 
subgroup  differences  will  also  reduce  validity.  Consequently,  in  the  context 
of  any  decision  about  how  to  reconstitute  a  test  battery,  a  number  of  very 
important  trade-offs  must  be  made. 

Modeling  First-Term  Attrition 

Attrition,  or  turnover,  from  an  organization  is  a  time-dependent 
outcome.  That  is,  the  relative  frequency  of  attrition  varies  over  time  in 
ways  that  appear  to  be  lawful  and  explainable,  rather  than  unsystematic  or 
random.  These  time-dependent  frequencies  invite  questions  about  the 
differential  prediction  of  attrition  across  time  and  whether  the  antecedents 
of  attrition  are  individual  differences,  situational  properties,  and/or  their 
interaction. 

The  individual  differences  component  of  these  time-dependent 
relationships  was  examined  by  using  event  history  analysis  to  model  the 
relationship  of  the  ASVAB  and  the  Experimental  Battery  predictors  with  first- 
tour  attrition  for  the  Longitudinal  Validation  Batch  A  sample  (i.e., 
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approximdtBly  28,000  soldisrs  from  nino  MOS  in  the  1986/87  accession  cohort). 
Event  history  models  look  at  the  degree  to  which  information  about  individual 
or  situational  differences  can  account  for  the  survivor  function  (the 
proportion  of  the  initial  sample  that  remains  in  the  organization  as  a 
function  of  the  passage  of  time)  and  the  hazard  function  (the  relative  rate  at 
which  people  are  leaving  the  organization  as  a  function  of  time). 

Given  a  baseline  survivor  function,  the  variable  that  currently  and 
historically  exerts  the  greatest  effect  on  attrition  rates  is  high  school 
graduation  status.  The  incremental  predictive  effects  of  the  ASVAB  and  each 
predictor  type  in  the  Experimental  Battery  were  analyzed  for  four  subgroups  of 
soldiers  defined  by  combat  vs.  non-combat  MOS  and  3-year  vs.  4-year 
enlistments.  The  ASVAB,  ABLE,  and  AVOICE  each  contributed  to  predicting  the 
attrition  over  and  above  the  effects  of  high  school  graduation  status. 

A  multiple  stepwise  regression  analysis  was  used  to  identify  the  best 
fitting  predictor  composite.  It  consists  of  high  school  graduation  status, 
five  ABLE  scales  (Nondelinquency,  Dominance,  Physical  Condition,  Self-Esteem, 
and  Social  Desirability),  and  the  Quantitative  composite  from  ASVAB.  Using 
progressively  higher  cut  scores  on  this  composite,  it  was  possible  to 
demonstrate  very  substantial  changes  in  the  survivor  function  for  all  four 
subgroups  of  the  LV  sample.  Perhaps  the  most  interesting  interaction  among 
the  predictors  was  that  the  ABLE  contributes  significantly  more  predictive 
validity  for  high  school  graduates  than  for  non-high  school  graduates,  which 
is  counter  to  the  conventional  wisdom  on  this  issue. 

In  general,  the  information  in  the  Experimental  Battery  provides 
significant  incremental  validity  to  high  school  graduation  status  as  a 
predictor  of  survival  rates  in  the  first  tour.  It  remains  to  be  determined 
whether  the  available  predictors  can  account  for  specific  differences  in  the 
attrition  rate  across  time.  For  example,  do  different  things  account  for 
attrition  during  training  as  contrasted  to  attrition  during  the  second  and 
third  years  of  a  4-year  term  of  enlistment? 

Job  Satisfaction  as  a  Predictor  of  Attrition  and  Reenlistment 

The  Army  Job  Satisfaction  Questionnaire  (AJSQ)  uses  22  items  in  a 
Likert-type  response  format  to  assess  six  facets  of  individual  job 
satisfaction.  Thirty-four  additional  items  reflect  the  individual's 
background  and  frame  of  reference.  The  AJSQ  was  administered  to  all 
individuals  in  the  LVI  and  LVII  samples.  For  both  samples,  scores  on  the  AJSQ 
were  related  to  the  available  performance  measures  and  to  the  soldiers'  stated 
intentions  to  attrit  or  to  reenlist. 

In  the  analysis  of  the  LVI  sample,  the  job  satisfaction  scores  were 
related  to  actual  attrition  and  actual  reenlistment.  In  this  sample  the  AJSQ 
was  administered  1  to  2  years  before  the  individual's  first  term  was  to  be 
concluded.  Consequently,  most  of  the  cohorts'  attrition  had  already  taken 
place  and  the  subsequent  attrition  rate  was  relatively  low. 

In  general,  the  correlations  of  job  satisfaction  with  job  performance 
and  reenlistment  were  significant  but  low.  For  performance,  the  Rs  were  about 
.20,  which  is  consistent  with  previous  Army  research.  The  job  satisfaction/ 
job  performance  relationship  in  the  military  Services  is  consistently  in  the 
.20  -  .25  range  while  the  analogous  results  in  the  non-military  sector  are  in 
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the  .10  -  .15  range.  The  correlations  with  attrition  were  lower,  but  the  low 
base'rate  for  attrition  tempers  any  conclusions  that  might  be  drawn.  The 
reldtionship  of  job  sdtisfdction  to  tho  stdtod  intent  to  reenlist  is  much 
higher,  and  intention  is  related  to  actual  reenlistment  to  a  significant 
degree. 

FUTURE  PLANS 

All  major  data  collections  that  were  designed  as  part  of  Project  A  and 
the  Career  Force  Project  have  been  completed;  all  major  data  files  are  edited 
and  in  place;  and,  with  the  completion  of  the  current  report,  all  "basic" 
predictor  and  criterion  analyses  have  also  been  completed.  What  remains  are  a 
series  of  maximizing/optimizing  analyses  as  regards  selection  efficiency  and 
classification  efficiency,  and  the  development  of  a  procedure  for  estimating 
potential  versus  actual  gains  from  classification  under  a  variety  of 
psychometric,  organizational,  and  labor  market  conditions.  The  planned 
sequence  of  events  is  as  follows. 

The  ASVAB  plus  the  entire  Experimental  Battery  will  be  used  to  identify 
the  45  equations  that  maximize  predictive  validity  (i.e.,  selection  validity) 
for  the  five  LVI  performance  factors  in  each  of  the  nine  Batch  A  MOS.  These 
45  equations  will  then  be  examined  in  terms  of  their  differential  validity 
across  MOS  and  across  performance  components  within  MOS.  One  major  question 
to  be  addressed  concerns  the  extent  to  which  these  45  predictor  equations  can 
be  collapsed  across  performance  components  within  MOS,  and  across  MOS  within 
performance  components,  without  losing  predictive  information.  Once  that 
question  is  answered,  the  next  issue  concerns  the  loss  of  predictive 
information  that  is  incurred  when  validity  weights,  synthetic  validity 
weights,  and  unit  weights  are  substituted  in  turn  for  the  sample-based 
regression  weights. 

To  the  fullest  extent  possible,  the  above  examination  of  the  content  and 
properties  of  the  optimal  selection  battery  for  each  major  performance 
component  in  each  MOS  will  also  be  carried  out  for  predicting  second-tour 
performance.  However,  the  LVII  sample  is  much  smaller  than  LVI,  which  places 
more  limitations  on  the  analyses. 

The  optimal  equations  identified  in  the  above  analyses  will  also  be 
evaluated  in  terms  of  their  selection  "fairness."  The  Career  Force  Project 
has  underway  a  major  analysis  of  prediction  fairness  that  uses  the  regression 
model  to  examine  virtually  all  predictor/criterion  combinations  in  each  MOS 
for  their  degree  of  under-  and  over-prediction  for  minority  groups.  The 
report  of  these  analyses  will  be  a  stand-alone  document  that  thoroughly 
portrays  the  extent  of  subgroup  differences,  predictive  fairness,  and 
predictive  validity  in  all  relevant  situations. 

The  optimal  equation  analyses  will  also  be  used  to  guide  the  analysis  of 
the  full  "Roll-Up  Model"  which  was  first  described  in  the  Statement  of  Work 
(SOW)  for  Project  A.  That  is,  what  is  the  effect  on  the  prediction  of  future 
performance  as  an  NCO  as  more  and  more  information  becomes  available  on  an 
individual?  Consequently,  the  validities  for  predicting  each  second-tour 
performance  component  will  be  analyzed  for  (a)  ASVAB  alone;  (b)  ASVAB  plus  the 
Experimental  Battery  (EB);  (c)  ASVAB  +  EB  +  end  of  training  performance  (EOT); 
(d)  ASVAB  +  EB  +  EOT  +  first-tour  performance  (LVI).  Each  set  of  information 
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win  be  analyzed  in  terms  of  its  unique  and  cumulative  contribution  to  the 
prediction  of  NCO  performance. 

Still  in  the  context  of  estimating  the  maximum  potential  selection 
validity  that  the  predictive  battery  provides,  two  additional  sets  of  analyses 
are  being  conducted.  One  is  directed  at  estimating  the  effects  on  validity  of 
using  empirical  keys  for  the  ABLE  and  AVOICE.  Each  inventory  is  being 
analyzed  against  the  Core  Technical  Performance  factor,  the  Leadership  factor, 
and  attrition  with  all  MOS  for  which  sample  sizes  are  sufficient.  Three 
different  types  of  keys  are  being  compared  and  they  will  be  evaluated  in  terms 
of  cross-validation  estimates,  cross-MOS  generalization/differentiation,  and 
cross-cohort  (i.e.,  CVI  vs.  LVI)  stability. 

The  second  set  of  analyses  concerns  correcting  all  major  Project 
A/Career  Force  validity  estimates  for  criterion  unreliability.  In  the  final 
analysis  we  would  like  to  make  comparative  statements  about  the  population 
validities  for  predicting  different  performance  components,  or  validities  for 
the  same  performance  component  when  it  is  measured  in  different  ways.  Doing 
this  requires  removing  differences  in  estimates  that  are  due  simply  to 
differences  in  criterion  reliabilities. 

The  culmination  of  the  Career  Force  Project  will  be  a  summarization  of 
its  implications  for  the  field  of  selection  and  classification  research  in 
general,  and  for  the  Army's  selection  and  classification  system  in  particular. 
In  this  regard,  the  Project  has  underway  a  major  effort  to  develop  a 
comprehensive  procedure  for  estimating  the  maximum  potential  gain  from 
classif ication-under  varying  conditions,  and  for  evaluating  the  extent  to 
which  alternative  operational  personnel  assignment  systems  can  realize  the 
maximum  potential  gain,  and  at  what  cost. 

We  will  attempt  to  integrate  previous  work  on  classification  efficiency 
estimation,  personnel  assignment  systems,  utility  measurement,  and 
cost/performance  trade-off  models  into  an  overall  design  for  an  evaluation 
test  bed  that  uses  the  extensive  Project  A/Career  Force  data  base  as  the 
specifications  for  its  latent  structure.  If  successful,  the  design  for  such  a 
test  bed  would  permit  systematic  evaluations  of  changes  in  the  test  battery, 
changes  in  costs,  changes  in  job  assignment  procedures,  and  so  forth  in  terms 
of  a  realistic  simulation  of  their  effects  on  meaningful  classification 
outcomes.  • 

Along  the  way,  a  number  of  critical  research  questions  must  be 
investigated.  For  example,  what  is  the  most  reasonable  way  to  select 
predictors  for  a  classification  battery?  Or,  what  is  the  most  appropriate 
index  with  which  to  represent  classification  gain?  Such  an  effort  is  a 
difficult  undertaking.  However,  it  represents  a  high  payoff  from  a  very 
comprehensive  series  of  projects  and  is  extremely  important,  for  both 
scientific  and  practical  purposes. 


251 


References 


Abrahams,  N.  M.,  Pass,  J.  J.,  Kusulas,  J.  W.,  Cole,  D.  R.,  &  Kieckhaefer, 

W.  F.  (1993).  Incremental  validity  of  experimental  computerized  tests 
for  predicting  training  criteria  in  military  technical  schools  (Contract 
N66001-90-D-9502,  Delivery  Order  7J13).  San  Diego:  Navy  Personnel 
Research  and  Development  Center. 

Ackerman,  P.  L.  (1989).  Within  task  intercorrelations  of  skilled  performance: 
Implications  for  predicting  individual  differences?  (A  comment  on  Henry 
&  Hulin,  1987).  Journal  of  Applied  Psychology,  74,  360-364. 

Allison,  P.  D.  (1984).  Event  history  analysis:  Regression  for  longitudinal 

event  data  (Sage  University  paper  series  on  quantitative  applications  in 
the  social  sciences.  No.  07-046).  Beverly  Hills,  CA:  Sage. 

Austin,  J.  T.,  Humphreys,  L.  G.,  Hulin,  C.  L.  (1989).  Another  view  of  dynamic 
criteria:  A  critical  reanalysis  of  Barrett,  Caldwell,  and  Alexander. 
Personnel  Psychology,  42,  583-596. 

Barrett,  G.  V.,  &  Alexander,  R.  A.  (1989).  Rejoinder  to  Austin,  Humphreys, 
and  Hulin:  Critical  reanalysis  of  Barrett,  Caldwell,  and  Alexander. 
Personnel  Psychology,  42 ,  597-612. 

Barrett,  G.  V.,  Alexander,  R.  A.,  &  Doverspike,  D.  (1992).  _  The  implications 
for  personnel  selection  of  apparent  declines  in  predictive  validities 
over  time:  A  critique  of  Hulin,  Henry,  and  Noon.  Personnel  Psychology, 
45,  601-617. 

Barrett,  G.  V.,  Caldwell,  M.  S.,  &  Alexander,  R.  A.  (1985).  The  concept  of 

dynamic  criteria:  A  critical  reanalysis.  Personnel  Psychology,  41- 
56. 


Bateman,  T.  S.,  &  Organ,  D.  W.  (1983).  Job  satisfaction  and  the  good  soldier: 
The  relationship  between  affect  and  employee  "citizenship."  Academy  of 
Management  Journal ,  26 .  587-595. 

Boesel,  D.,  &  Johnson,  K.  (1984).  Why  Service  members  leave  the  military: 
Review  0.  of  the  literature  and  analysis.  Washington,  DC:  Defense 
Manpower  Data  Center. 

Borman,  W.C.,  &  Bleda,  P.R.  (1978).  Measuring  motivation  and  job  satisfaction 
in  a  military  context  (ARI  TP  309).  Alexandria,  VA:  U.S.  Army  Research 
Institute  for  the  Behavioral  and  Social  Sciences. 

Borman,  W.  C.,  &  Motowidlo,  S.  J.  (1993).  Expanding  the  criterion  domain  to 
include  elements  of  contextual  performance.  In  N.  Schmitt  and  W.  C. 
Borman  (Eds.),  Personnel  selection  in  organizations.  San  Erancisco: 
Jossey-Bass . 

« 

Brogden,  H.  (1959).  Efficiency  of  classification  as  a  function  of  number  of 
jobs,  percent  rejected,  and  the  validity  and  intercorrelation  of  job 
performance  estimates.  Educational  and  Psychological  Measurement, 

19(2),  181-190. 


253 


Campbell,  C.  H.,  Ford,  P.,  Rumsey,  M.  G.,  Pulakos,  E.  D.,  Borman,  W.  C., 

Felker,  0.  B.,  de  Vera,  M.  V.,  &  Riegelhaupt,  B.  J.  (1990).  Development 
of  multiple  job  performance  measures  in  a  representative  sample  of  jobs. 
Personnel  Psychology,  43,  277-300. 

Campbell,  J.  P.  (Ed.)  (1987).  Improving  the  selection,  classification,  .and 

utilization  of  Army  enlisted  personnel:  Annual  report,  1986  fiscal  year 
(ARI  Technical  Report  813101).  Alexandria,  VA:  U.S.  Army  Research 
Institute  for  the  Behavioral  and  Social  Sciences. 

Campbell,  J.  P.  (Ed.)  (1989).  Improving  the  selection,  classification,  and 

utilization  of  Army  enlisted  personnel;  Annual  report,  1987  fiscal  year 
(ARI  Technical  Report  862).  Alexandria,  VA:  U.S.  Army  Research 
Institute  for  the  Behavioral  and  Social  Sciences.  (ADA  219  046) 

Campbell,  J.  P.  (Ed.)  (1991).  Improving  the  selection,  classification,  and 

utilization  of  Army  enlisted  personnel:  Annual  report,  1988  fiscal  year 
(ARI  Research  Note  91-34).  Alexandria,  VA:  U.S.  Army  Research 
Institute  for  the  Behavioral  and  Social  Sciences.  (ADA  233  750) 

Campbell,  J.  P.,  McHenry,  J.  J.,  &  Wise,  L.  L.  (1990).  Modeling  job 

performance  in  a  population  of  jobs.  Personnel  Psychology,  313-333. 

Campbell,  J.  P.,  &  Oppler,  S.  (1990).  Modeling  of  second-tour  performance. 

In  J.  P.  Campbell  &  L.  M.  Zook  (Eds.),  Building  and  retaining  the  Career 
Force:  New  procedures  for  accessing  and  assigning  Army  enlisted 
personnel  -  Annual  Report.  1990  fiscal  year  (ARI  Technical  Report  952). 
Alexandria,  VA:  U.S.  Army  Research  Institute  for  the  Behavioral  and 
Social  Sciences.  (ADA  252  675) 

Campbell,  J.  P.,  &  Zook,  L.  M.  (Eds.)  (1990).  Building  and  retaining  the 

Career  Force:  New  procedures  for  accessing  and  assigning  Army  enlisted 
personnel  -  Annual  Report,  1990  fiscal  year  (ARI  Technical  Report  952). 
Alexandria,  VA:  U.S.  Army  Research  Institute  for  the  Behavioral  and 
Social  Sciences.  (ADA  252  675) 

Campbell,  J.  P.,  &  Zook,  L.  M.  (Eds.)  (1991).  Improving  the  selection, 
classification,  and  utilization  of  Army  enlisted  personnel:  Final 
Report  on  Project  A  (ARI  Research  Report  1597).  Alexandria,  VA:  U.S. 
Army  Research  Institute  for  the  Behavioral  and  Social  Sciences.  (ADA  242 
921) 

Campbell,  J.  P.,  &  Zook,  L.  M.  (Eds.)  (1994a).  Building  and  retaining  the 

Career  Force:  New  procedures  for  accessing  and  assigning  Army  enlisted 

personnel  -  Annual  Report,  1991  fiscal  year  (ARI  Research  Note  94-10). 

Alexandria,  VA:  U.S.  Army  Research  Institute  for  the  Behavioral  and 
Social  SciencesT  (ADA  278  726) 

Campbell,  J.  P.,  &  Zook,  L.  M.  (Eds.)  (1994b).  Building  and  retaining  the 

Career  Force:  Nev/  procedures  for  accessing  and  assigning  Army  enlisted 

personnel  -  Annual  Report,  1992  fiscal  year  (ARI  Research  Note  94-27). 

Alexandria,  VA:  U.S.  Army  Research  Institute  for  the  Behavioral  and 
Social  Sciences.  (AD  B1S8  259) 


254 


Campbell,  R.  V.  (1985).  Scorer  training  materials  (ARI  RS-WP-85). 

Alexandria,  VA:  U.S.  Army  Research  Institute  for  the  Behavioral  and 
Social  Sciences. 

Carter,  G.  W.  (1991).  A  study  of  relationships  between  measures  of  individual 
differences  and  job  satisfaction  among  U.S.  Army  enlisted  personnel. 
Unpublished  doctoral  dissertation,  University  of  Minnesota. 

Chapman,  L.  J.,  &  Chapman,  J.  P.  (1969).  Illusory  correlation  as  an  obstacle 
to  the  use  of  valid  diagnostic  signs.  Journal  of  Abnormal  Psychology, 
74,  271-280. 

Claugy,  J.  G.  (1978).  Multiple  regression  and  validity  estimation  in  one 
sample.  Applied  Psychological  Measurement,  2,  295-801. 

Collett,  D.  (1991).  Modelling  binary  data.  London:  Chapman  &  Hall. 

Cox,  D.  R.  (1972).  Regression  models  and  life  tables.  Journal  of  the  Royal 
Statistical  Society,  34,  187-202. 

Darlington,  R.  B.  (1972).  Multiple  regression  in  psychological  research  and 
practice.  Psychological  Bulletin,  ^( 3_) ,  161-182. 

Department  of  Defense  (1982).  Profile  of  American  youth:  1980  nationwide 
administration  of  the  Armed  Services  Vocational  Aptitude  Battery. 
Washington,  DC:  Office  of  the  Assistant  Secretary  of  Defense  (Manpower, 
R&serye  Affairs,  and  Logistics). 

Department  of  Defense  (1985,  May).  Defense  manpower  Quality  (Volumes  I-III). 
Report  to  the  House  and  Senate  Committees  on  Armed  Services. 

Washington,  DC:  Office  of  the  Assistant  Secretary  of  Defense  (Manpower, 
Installations,  and  Logistics). 

Equal  Opportunity  Employment  Commission,  U.S.  Civil  Service  Commission, 

Department  of  Labor,  &  Department  of  Justice  (1978,  August).  Uniform 
guidelines  on  employee  selection  procedures.  43  Fed.  Reg.  166,  38290- 
38309. 

Etheridge,  R.  M.  (1989).  Family  factors  affecting  retention:  A  reyiew  of  the 
1 iterature  (ARI  Research  Report  1511).  Alexandria,  VA:  U.S.  Army 
Research  Institute  for  the  Behavioral  and  Social  Sciences. 

Farkas,  A.  J.,  &  Tetrick,  L.  E.  (1989).  A  three-wave  longitudinal  analysis  of 
the  causal  ordering  of  satisfaction  and  commitment  on  turnover 
decisions.  Journal  of  Applied  Psychology.  74,  855-868. 

Fichman,  M.  (1988).  Motiyational  consequences  of  absence  and  attendance: 

Proportional  hazard  estimation  of  a  dynamic  motiyation  model.  Journal 
of  Applied  Psychology,  73,  119-134. 

Fichman,  M.  (1989).  Attendance  makes  the  heart  grow  fonder:  A  hazard  rate 
approach  to  modeling  attendance.  Journal  of  Applied  Psychology,  2^, 
325-335. 


255 


Flyer  E.  S.  (1959).  Factors  relating  to  discharge  for  unsuitability  among 

1956  airmen  accessions  to  the  Air  Force  (WADC-TN-59-201 ) .  Lackland  Air 
Force  Base,  TX:  Personnel  Research  Laboratory. 

Goldberg,  L.  R.  (1981).  Language  and  individual  differences:  The  search  for 
universals  in  personality  lexicons.  In  L.  Wheeler  (Ed.),  Review  of 
personality  and  social  psychology  (Vol.  2,  pp.  141-165).  Beverly  Hills, 
CA:  Sage. 

Hanson,  M.  A.,  &  Borman,  W.  B.  (in  press).  Development  and  construct 
validation  of  the  Situational  Judgment  Test  (SJT)  (ARI  Technical 
Report).  Alexandria,  VA:  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences. 

Hanson,  M.  A.,  Campbell,  J.  P.,  &  McKee,  A.  S.  (1994).  Development  of  the 

longitudinal  validation  sample  second-tour  performance  model.  In  J.  P. 
Campbell  &  L.  M.  Zook  (Eds.),  Building  and  retaining  the  Career  Force: 
New  procedures  for  accessing  and  assigning  Army  enlisted  personnel.,^ 
Annual  Report.  1992  fiscal  year  (ARI  Research  Note  94-27).  Alexandria, 
VA:  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social 
Sciences.  (AD  B188  259) 

Harrell,  F.  (1980).  The  PHGLM  procedure.  In  SAS  Supplemental  Library  User's 
Guide  (pp.  119-131).  Cary,  NC:  SAS  Institute. 

Harrison,  D.  A.,  &  Hulin,  C.  L.  (1989).  Investigations  of  absenteeism:  Using 
event  history  models  to  study  the  absence-taking  process.  Journal  of. 
Applied  Psychology,  74,  300-316. 

Henry,  R.  A.,  &  Hulin,  C.  L.  (1987).  Stability  of  skilled  performance  across 
time:  Some  generalizations  and  limitations  on  utilities.  Journal  of 
Applied  Psychology,  72,  457-462. 

Hiller,  J.R.  (1982).  Analysis  of  second  term  reenlistment  behavior.  Santa 
Monica,  CA:  Rand  Corporation. 

Horn,  P.  W.,  &  Hulin,  C.  L.  (1981).  A  competitive  test  of  the  prediction  of 
reenlistment  by  several  models.  Journal  of  Applied  Psychology,  66_, 
23-39. 


Horn,  P.  W.,  Katerberg,  R.,  &  Hulin,  C.  L.  (1979).  Comparative  examination  of 
three  approaches  to  the  prediction  of  turnover.  Journal  of  Applied 
Psychology,  64 ,  280-290. 

Horn,  P.  W.,  Prussia,  G.  E.,  &  Griffeth,  R.  W.  (1992).  A  meta-analytical 

structural  equations  analysis  of  a  model  of  employee  turnover.  JoucnaX 
of  Applied  Psychology.  77 .  890-909. 

Hough,  L.  M.  (1992).  The  "Big  Five"  personality  variables  -  construct 
confusion:  Description  versus  prediction.  Human  Performance, 

139-155. 


256 


Hough,  L.  M.,  Barge,  B.  N.,  &  Kamp,  J.  D.  (1987).  Non-cognitive  measures: 

Pilot  testing.  In  N.  G.  Peterson  (Ed.),  Development  and  field  test  of 
the  trial  battery  for  Project  A  (ARI  Technical  Report  739).  Alexandria, 
VA:  U.  S.  Army  Research  Institute  for  the  Behavioral  and  Social 
Sciences . 

Hough,  L.  M.,  Eaton,  N.  K.,  Dunnette,  M.  D.,  Kamp,  J.  D.,  &  McCloy,  R.  A. 

(1990).  Criterion-related  validities  of  personality  constructs  andthe 
effect  of  response  distortion  on  those  validities.  Journal  of  Applied 
Psychology.  75(5),  581-595. 

Hulin,  C.  L.,  Henry,  R.  A.,  &  Noon,  S.  L.  (1990).  Adding  a  dimension:  Time 
as  a  factor  in  the  general izabi 1 ity  of  predictiye  relationships. 
Psychological  Bulletin.  107 .  328-340. 

laffaldano,  M.  T.,  &  Muchinsky,  P.  M.  (1985).  Job  satisfaction  and  job 

performance:  A  meta-analysis.  Psychological  Bulletin,  97.,  251-273. 

Ironson,  G.  H.,  &  Smith,  P.  C.  (1981).  Anchors  away  -  The  stability  of 
meaning  when  their  location  is  changed.  Personnel  Psychology,  M. 
249-262. 

Jensen,  A.  R.  (1985).  The  nature  of  black-white  differences  on  yarious 

psychometric  tests.  The  Behavioral  and  Brain  Sciences,  2^,  297-300. 

Johnson,  C.  D.,  &  Zeidner,  J.  (1990).  Classification  utility: _ Measuring  and. 

improyjng  benefits  in  matching  personnel  to  jobs  (IDA  Paper  P-2240). 
Alexandria,  VA:  Institute  for  Defense  Analysis. 

Joreskog,  K.  G.,  &  Sorbom,  D.  (1986).  LISREL  VI:  Analysis  of  linear 
structural  relationships  by  the  method  of  maximum  likelihood. 
Morresyille,  IN:  Scientific  Software. 

Joreskog,  K.  G.,  &  Sorbom,  D.  (1989).  LISREL  7:  A  guide  to  the  program  and 
appl ications  (2nd  ed.).  Chicago:  SPSS. 

Kalbfleisch,  J.  D.,  &  Prentice,  R.  L.  (1980).  The  statistical  analysis  of 
failure  time  data.  New  York:  John  Wiley  and  Sons. 

Kieckhaefer,  W.  F.,  Ward,  D.  G.,  Kusulas,  J.  W.,  Cole,  D.  R.,  Rupp,  L.  .M.,  & 
May,  M.  H.  (1992).  Criterion  development  for  18  technical  training 
schools  in  the  Navv.  Army,  and  Air  Force  (Contract  #  N66001-90-D-9502, 
Delivery  Order  7J08).  San  Diego:  Navy  Personnel  Research  and 
Development  Center. 

Klein,  S.,  Hawes-Dawson ,  J.,  &  Martin,  T.  (1991).  Why  recruits  separate  early 
(R-3980-FMP) .  Santa  Monica,  CA:  Rand  Corporation. 

Knapp,  D.  J.  (1993,  August).  Alternative  conceptualizations  of  turnover.  In 
J.P.  Campbell  (Chair),  Prediction  of  turnover  in  a  longitudinal  sample 
using  event  history  analysis.  Symposium  conducted  at  the  conyention  of 
the  American  Psychological  Association,  Toronto. 

LaRocco,  J.  M.,  Pugh,  W.  M.,  &  Gunderson,  E.  K.  (1977).  Identifying 

determinants  of  retention  decisions.  Personnel  Psychology,  30,  199-215. 


257 


Laurence,  J.  H.  (1993).  Education  standards  and  military  selection:  From  the 
beginning.  In  T.  Trent  &  J.  H.  Laurence  (Eds.),  Adaptability  screening 
for  the  Armed  Forces  (pp.  1-40).  Washington,  DC:  Office  of  the 
Assistant  Secretary  of  Defense. 

Lawless,  J.  F.  (1982).  Statistical  models  and  methods  for  lifetime  data. 

New  York:  John  Wiley  and  Sons. 

Linn,  R.  L.  (1968).  Range  restriction  problems  in  the  use  of  self  selected 
groups  for  test  validation.  Psychological  Bulletin,  6^,  69-73. 

Lord,  F.  M.,  &  Novick,  M.  R.  (1968).  Statistical  theories  of  mental  tes_t 
scores .  Reading  MA:  Addison-Wesley. 

McBride,  J.  R.  (1993).  Compensatory  screening  model  development.  In  T.  Trent 
&'j.  H.  Laurence  (Eds.),  Adaptability  screening  for  the  Armed  Forces 
(pp.  1-40).  Washington,  DC:  Office  of  the  Assistant  Secretary  of 
Defense. 

McCloy,  R.  A.  (1993).  An  overview  of  survival  analysis.  Paper  presented  at 
the  conference  of  the  American  Psychological  Association,  Toronto. 

McCloy,  R.  A.,  Harris,  D.  A.,  Barnes,  J.  D.,  Hogan,  P.  F.,  Smith,  D.  A., 

Clifton,  D.,  &  Sola,  M.  (1992).  Accession  Quality,  job  performance,  and_ 
cost:  A  cost-performance  tradeoff  model  (FR-PRD-92-11 ) .  Alexandria, 

VA:  Human  Resources  Research  Organization. 

McCloy,  "r.  a.,  &  Oppler,  S.  H.  (1990).  End-of-training  measures.  In  J.  P. 

Campbell  &  L.  M.  Zook  (Eds.),  Building  and  retaining  the  Career  Force: 
New  procedures  for  accessing  and  assigning  Army  enlisted  personnel  - 
Annual  Report,  1990  fiscal  year  (ARI  Technical  Report  952).  Alexandria, 
VA:  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social 
Sciences.  (ADA  252  675) 

McEvoy,  G.  M.,  &  Cascio,  W.  F.  (1987).  Do  good  or  poor  performers  leave?  A 
meta-analysis  of  the  relationship  between  performance  and  turnover. 
Academy  of  Management  Journal.  30,  744-762. 

McHenry,  J.  J.,  Hough,  L.  M. ,  Toquam,  J.  L.,  Hanson,  M.  A.,  &  Ashworth,  S. 

(1990).  Project  A  validity  results:  The  relationship  between  predictor 
and  criterion  domains.  Personnal  Psychology,  £3,  335-354. 

Miller,  H.  E.,  Katerberg,  R.,  &  Hulin,  C.  L.  (1979).  Eyaluation  of  the 

Mobley,  Horner,  and  Hollingsworth  model  of  employee  turnoyer.  Journal 
of  Applied  Psychology.  64.  509-517. 

Mobley,  W.  H.,  Hand,  H.  H.,  Baker,  R.  L.,  &  Meglino,  B.  M.  (1979).  Conceptual 
and  empirical  analysis  of  military  recruit  training  attrition.  Journal 
of  Applied  Psychology.  64 .  10-18. 

Morita,  J.  G.,  Lee,  T.  W.,  &  Mowday,  R.  T.  (1989).  Introducing  survival 
analysis  to  organizational  researchers:  A  selected  application  to 
turnover  research.  Journal  of  Applied  Psychology,  74 ,  280-292. 


258 


Motowidlo,  S.  J.,  Dowell,  B.  E.,  Hopp,  M.  A.,  Borman,  W.  C.  Johnson,  P.D.  & 
Dunnette,  M.  S.  (1976).  Motivation,  satisfaction,  and  morale  in  Army 
careers:  A  review  of  theory  and  measurement  (ARI  Technical  Report 
75-/\7 ) .  Alexandria,  VA:  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences. 


Motowidlo,  S.  J. 
soldiers ' 
157-166. 


&  Lawton,  G.  W.  (1984).  Affective  and  cognitive  factors 
reenlistment  decisions.  Journal  of  Applied  Psychology,  6^, 


in 


Norman,  W.  T.  (1963).  Toward  an  adequate  taxonomy  of  personality  attributes: 
Replicated  factor  structure  in  peer  nomination  personality  ratings. 
Journal  of  Abnormal  and  Social  Psychology,  574-583. 

Nunnally,  J.  C.  (1967).  Psychometric  theory.  New  York:  McGraw-Hill. 

Oppler,  S.  H.,  Childs,  R.  A.,  &  Peterson,  N.  G.  (1994).  Development  of  the 
longitudinal  validation  sample  first-tour  performance  model.  In  J.  P. 
Campbell  &  L.  M.  Zook  (Eds.),  Building  and  retaining  the  Career  Force.L, 
New  procedures  for  accessing  and  assigning  Army  enlisted  personnel  - 
Annual  Report.  1991  fiscal  year  (ARI  Research  Note  94-10).  Alexandria, 
VA:  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social 
Sciences.  (ADA  278  726) 

Oppler,  S.  H,  McCloy,  R.  A.,  &  Peterson,  N.  G.  (1994).  Prediction  of 
performance  in  training.  In  J.  P.  Campbell  &  L.  M.  Zook  (Eds.), 

Building  and  retaining  the  Career  Force:  New  procedures  for  accessing 

and  assigning  Army  enlisted  personnel  -  Annual _ Report,^ — 1991  fiscal — 

Chapter  3  (ARI  Research  Note  94-10).  Alexandria,  VA:  U.S.  Army 
Research  Institute  for  the  Behavioral  and  Social  Sciences.  (ADA  278  726) 


Oppler,  S.  H.,  Peterson,  N.  G.,  &  Russell,  T.  (1994).  Basic  validation 
results  for  the  LVI  sample.  In  J.  P.  Campbell  &  L.  M.  Zook  (Eds.), 
Building  and  retaining  the  Career  Force:  New  procedures  for  accessing 
and  assigning  Army  enlisted  personnel  -  Annual  report, — 1991  fiscal — year 
(ARI  Research  Note  94-10).  Alexandria,  VA:  U.S.  Army  Research 
Institute  for  the  Behavioral  and  Social  Sciences.  (ADA  278  72Gj 


Organ,  D.  W.  (1988).  Organizational  citizenship  behavior: _ The  good  soldier 

syndrome.  Lexington,  MA:  D.  C.  Heath. 

Peterson  N.  G.,  Hough,  L.  M.,  Dunnette,  M.  D.,  Rosse,  R.  L.,  Houston,  J.  S., 
Toquam,  J.  L.,  &  Wing,  H.  (1990).  Project  A:  Specification  of  the 
predictor  domain  and  development  of  new  selection/classification  tests. 
Personnel  Psychology.  43 ,  247-276. 

Peterson,  N.  G.,  Oppler,  S.  H.,  Sager,  C.  E.,  &  Rosse,  R.  L.  (1993).  Anal ysvs 
of  the  Enhanced  Computer  Administered  Test  Battery:  ^  An  evaluation  of 
potential  revisions  and  additions  to  the  Armed  Services  Vocational, 
Aptitude  Battery  (Draft  report  prepared  for  the  Selection  and 
Classification  for  Critical  MOS  Project).  Washington,  DC:  American 
Institutes  for  Research.  (ADA  252  675) 


259 


Peterson,  N.  G.,  Russell,  T.  L.,  Hallam,  G.,  Hough,  L.  M.,  Owens-Kurtz,  C.  K., 
Gialluca,  K.,  and  Kerwin,  C.  (1990).  Analysis  of  the  Experimental 
Predictor  Battery:  LV  Sample.  In  J.  P.  Campbell  &  L.  M.  Zook  (Eds.), 
Building  and  retaining  the  Career  Force:  New  procedures  for  accessing 
and  assigning  Army  enlisted  personnel  -  Annual  Report,  1990  fiscal  year. 
Chapter  3  (ARI  Technical  Report  952).  Alexandria,  VA:  U.S.  Army 
Research  Institute  for  the  Behavioral  and  Social  Sciences.  (ADA  252  675) 

Peterson,  N.,  Russell,  T.,  Hallam,  G.,  Hough,  L.,  Ov/ens-Kurtz ,  C.,  Gialluca, 
K.,  &  Kerwin,  K.  (1990).  Analysis  of  the  experimental  predictor 
battery:  LV  sample.  In  J.  P.  Campbell  &  L.  M.  Zook  (Eds.),  Bui Iding 
and  retaining  the  Career  Force:  New  procedures  for  accessing  and 
assigning  Army  enlisted  personnel  -  Annual  report,  1990  fiscal  year  (ARI 
Technical  Report  952,  pp.  73-199).  Alexandria,  VA:  U.S.  Army  Research 
Institute  for  the  Behavioral  and  Social  Sciences.  (ADA  252  675) 

Petty,  M.  M.,  McGee,  G.  W.,  &  Cavender,  J.  W.  (1984).  A  meta-analysis  of  the 
relationships  between  individual  job  satisfaction  and  individual 
performance.  Academy  of  Management  Review.  £,  712-723. 

Podsakoff,  P.  M.,  &  Williams,  L.  (1986).  The  relationship  between  job 

performance  and  job  satisfaction.  In  E.  A.  Locke  (Ed.),  General izing 
from  laboratory  to  field  settings  (pp.  207-253).  Lexington,  MA; 
Lexington  Books. 

Ree,  J.  R.,  &  Earles,  J.  A.  (1991).  Predicting  training  success:  Not  much 
more  than  g.  Personnel  Psychology,  44,  321-332. 

Research  Triangle  Institute  (1988).  "Family  research  program  and  the  Project 
A  data  base"  (RTI/3795-44  WP).  Draft  working  paper.  Research  Triangle 
Park,  NC:  Author. 

Royle,  M.  H.,  &  Robertson,  D.  W.  (1980).  Job  satisfaction  measures  as 

predictors  of  retention  for  Navy  enlisted  personnel  (TR  81-2).  San 
Diego,  CA:  Navy  Personnel  Research  and  Development  Center. 

Rozeboom,  W.  W.  (1978).  Estimation  of  cross-validated  multiple  correlation: 

A  clarification.  Psychological  Bulletin,  85,  1348-1351. 

SAS  Institute  (1992).  The  PHREG  procedure.  In  SAS  Technical  Report  P-229, 
SAS/STAT  software:  Changes  and  enhancements,  release  6.07  (pp.  433- 
479).  Cary,  NC:  Author. 

Scholarios,  T.  M.  (1990).  Maximizing  potential  classification  efficiency: 
Selection  of  predictor  measures  based  on  alternative  psychometric 
indices.  Unpublished  doctoral  dissertation,  George  Washington 
University. 

Singer,  J.  D.,  &  Willett,  J.  B.  (1991).  Modeling  the  days  of  our  lives: 

Using  survival  analysis  when  designing  and  analyzing  longitudinal 
studies  of  duration  and  the  timing  of  events.  Psychological  Bulletin. 
110(2).  268-290. 


260 


Singer 


J.  D.,  &  Willett,  J.  B.  (1993).  It's  about  time:  Using  discrete-time 
survival  analysis  to  study  duration  and  the  timing  of  events,  I, 

of  Educational  Statistics,  18(2),  155-195. 


Smith 


C.  A.  Organ,  0.  W.,  &  Near,  J.  P.  (1983).  Organizational  citizenship 
behavior:  Its  nature  and  antecedents.  Journal  of  Applied  Psychology, 
M(i).  653-663. 


Smith,  P.C.,  Kendall,  L.M.,  &  Hulin,  C.L.  (1969).  Measurement_of 
in  work  and  retirement.  New  York:  Rand  McNally. 


satisfaction 


Statman  M.  A.  (1993).  ImprovinQ  the  effectiveness  of  employment  testing. 
through  classification:  Alternative  methods  of  developing  test 
composites  for  optimal  job  assignment  and  vocational  counseling. 
Submitted  doctoral  dissertation,  George  Washington  University. 


Stein  C.  (1960).  Multiple  regression.  In  I.  Olkin  (Ed.),  Contributions  t^ 
'probability  and  statistics.  Stanford,  CA:  Stanford  University  Press. 

Steinhaus  S.  0.  (1988).  Predicting  military  attrition  from  educational  and 
biographical  information  (FR-PRD-88-Q6) .  Alexandria,  VA:  Human 
Resources  Research  Organization. 

Tett  R.  P.,  &  Meyer,  J.  P.  (1993).  Job  satisfaction,  organizational 

commitment,  turnoyer  intention,  and  turnover:  Path  analyses  based  on 
meta-analytic  findings.  Personnel  Psychology,  259-293. 

Trent  T.  (1993).  The  Armed  Seryices  Applicant  Profile  (ASAP).  In  T.  Trent 
'and  J.  H.  Laurence  (Eds.),  Adaptability  screening  for  the  Armed  Forces 
(pp.  71-99).  Washington,  DC:  Office  of  the  Assistant  Secretary  of 
Defense. 


Vroom,  V.  H.  (1964).  Work  and  motiyation.  New  York:  John  Wiley  &  Sons. 

Waters,  B.  K.,  Barnes,  J.  D.,  Foley,  P.,  Steinhaus,  S.  D.,  &  Brown,  D.  C. 
(1988).  Estimating  the  reading  skills  of  military  applicants: 
Development  of  an  ASVAB  to  RGL  conyersion  table  (FR-PRD-88-22) . 
Alexandria,  VA:  Human  Resources  Research  Organization. 

Weiss,  D.  J.,  Dawis,  R.  V.,  England,  G.  W.,  &  Lofquist,  L.  H.  (1967).  Manual 
for  the  Minnesota  Satisfaction  Questionnaire.  Minnesota  Studies.in 
Vocational  Rehabilitation,  71. 

White  L.  A.  (1994).  Deyelopment  of  composite  scores  for  Assessment  of  the 
'Background  and  Life  Experiences  (ABLE)  Instrument.  In  J.  P.  Campbell  & 
L.  M.  Zook  (Eds.),  Building  and  retaining  the  Career  Force: — 
procedures  for  accessing  and  assigning  Army  enlisted  personnel  -  Annual 
Report,  1992  fiscal  year  (ARI  Research  Note  94-27,  pp.  25-32). 
Al6Xdn(jris,  VA:  U.  S.  Army  ResBdrch  Institute  for  the  Behavioral  and 

Social  Sciences.  (AD,  B188  259) 


Wh  i  te 


L.  A.,  Nord,  R.  D.,  Mael,  F.  A.,  &  Young,  M.  C.  (1993).  The  Assessment 
of  Background  and  Life  Experiences  (ABLE).  In  T.  Trent  &  J.  H.  Laurence 
(Eds.),  Adaptability  screening  for  the  Armed  Forces  (pp.  101-162). 
Washington,  DC:  Office  of  the  Assistant  Secretary  of  Defense. 


261 


Willett,  J.  B.,  &  Singer,  J.  D.  (1993).  It's  de.ia.r^u 
multiple-spell  discrete-time  survival  analysis, 
for  publication. 


al 1  over  again:  Using 
Manuscript  submitted 


Willett,  J.  B.,  &  Singer,  J.  D.  (in  press).  Investigating  onset,  cessation, 
relapse  and  recovery:  Why  you  should,  and  how  you  can,  use  discrete¬ 
time  survival  analysis.  Journal  of  Consulting  and  Clinical  Psychology. 

Wise  L.  L.,  McHenry,  J.  J.,  &  Campbell,  J.  P.  (1990).  Identifying  optimal 
predictor  composites  and  testing  for  general izabi 1 ity  across  jobs  and 
performance  factors.  Personnel  Psychology  43,  355-366. 

Wise,  L.  L.,  McHenry,  J.  J.,  &  Young,  W.  Y.  (1986).  Project  A  Concurrent 
Validation:  Treatment  of  Missing  Data.  Alexandria,  VA:  U.5.  Army 
Research  Institute  for  the  Behavioral  and  Social  Sciences. 


Wise,  L.  L.,  &  McLaughlin,  D.  H.  (1980).  Guidebook  for  the  imputation  of 
missing  data.  Palo  Alto,  CA:  American  Institutes  for  Research. 

Wise,  L.  L.,  Peterson,  N.  G.,  Hoffman,  R.  G.,  Campbell,  J.  P.,  ^ 

M  nqqil.  The  Armv  Synthetic  Validity  Project: Report  of  Phase  Uj 

results,  Vol.  I  (ARI  Technical  Report  922).  Alexandria,  VA:  U.S.  Army 
Research  Institute  for  the  Behavioral  and  Social  Sciences.  (i\q  A235  635) 


Wise,  L.  L.,  Welsh,  J.,  Grafton,  F.,  Foley,  P.,  Earles,  J.,  Swain,  L.,  & 

Divgi  D.  R.,  (1992).  Sensitivity  and  fairness  of  the  Armed  Services 
Vocational  Aptitude  Battery  (ASVAB)  technical  composites.  Monterey  CA: 
Defense  Manpower  Data  Center,  Personnel  Testing  Division. 

Wolfe,  J.  H.,  Alderton,  D.  L.,  Larson,  G.  E.,  &  Held,  J.  D.  (in  preparation). 
“ I ncreni6n td  1  vdlidity  of  Enhdncod  Computor  Administored  iGsting  (LLAij 
(Draft  Technical  Report).  San  Diego,  CA:  Navy  Personnel  Research  and 
Developmental  Center. 

Young,  M.  C.,  White,  L.  A.,  &  Oppler,  S.  H.  (1992).  Effects  of  coaching  on 

validity  of  a  self-report  temperament  measure.  Proceedings  of  the  i4tn 
Annual  Conferences  of  the  Military  Testing  Association  (pp.  188-193). 
San  Diego. 

Youngblood,  S.  A.,  Mobley,  W.  H.,  &  Meglino,  B.  M.  (1983).  A  longitudinal 
analysis  of  the  turnover  process.  Journal  of  Applied  Psvcholo_qy., 
507-516. 


262 


Appendix  A 


VALIDITIES  OF  THE  TOP  20  POTENTIAL  TEST  BATTERIES 
RANKED  ACCORDING  TO  EACH  PERFORMANCE  INDEX 
WITHIN  EACH  CRITERION  AND  EACH  TIME  INTERVAL 


Note:  For  each  battery  the  tests  are  listed  in  an  arbitrary  order  (i.e.,  the 
order  provides  no  information  about  the  relative  contribution  of 
each  test). 

All  variance/covariance  matrices  used  to  calculate  validity  coefficients 
are  corrected  for  multivariate  range  restriction  (Lord  &  Novick, 
1968,  p.l47)  before  averaging. 

Each  validity  estimate  is  adjusted  using  Rozeboom  formula  #8  before 
averaging. 

Means  are  computed  as  unweighted  (i.e.,  equally  weighted)  means  across 
nine  MOS.  Corresponding  SDs  are  "unbiased"  estimates,  based  on 
the  same  nine  MOS. 

Discriminant  validity  is  the  difference  between  the  unweighted  mean 

validity  and  the  mean  correlation  of  the  MOS-specific  equations 
with  the  other  eight  MOS. 

Brogden  index  of  classification  efficiency  is  the  mean  absolute  validity 
times  the  square-root  of  (one  minus  the  mean  correlation  among  the 
predicted  scores). 

All  test  battery  times  are  computed  with  3  minutes  added  for  each  test, 
plus  20  minutes  of  start-up  time. 

Subgroup  difference  estimates  =  difference  between  mean  composite  of  the 
majority  group  and  the  mean  composite  of  the  minority  group 
divided  by  the  majority  group's  standard  deviation;  these 
estimates  are  based  on  means  and  variances  that  have  been 
corrected  for  range  restriction  (the  means,  variances,  and 
covariances  for  the  10  ASVAB  scores  for  majority  and  minority 
groups  were  obtained  from  the  1980  Youth  Population  and  the 
remaining  variances,  covariances,  and  means  are  obtained  by  range 
restriction  corrections). 
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Predictor  Test  Names 


ASAR  Arithmetic  Reasoning 

ASWK  Word  Knowledge 

ASAS  Auto  &  Shop  Information 

ASCS  Coding  Speed 

ASEI  Electronics  Information 

ASGS  General  Science 

ASMC  Mechanical  Comprehension 

ASMK  Mathematical  Knowledge 

ASNO  Numerical  Operations 

ASPC  Paragraph  Comprehension 

SPAO  Assembling  Objects 

SPOR  Orientation 

SPRS  Reasoning 

CMST  Short-Term  Memory 

CMTI  Target  Identification 

CMIT  One-Hand  Tracking 

CM2T  Two-Hand  Tracking 


Appendix  Table  Headings 

Validity  Estimates 
Discriminant  Validity 
Brogden  Index 

White-Black  Subgroup  Differences 
White-Hispanic  Subgroup  Differences 
Male-Female  Subgroup  Differences 


A-2 


K 

C3 

1 


c; 

(j 

c 


fO 

UJ 


■o 

u 

o 

u 

u 

< 

■S 


a. 

o 


"S  1c 
i£ 

5  a; 
c:  •<-> 
^  c 


—  o 

rO  1-H 
U  I 

•I-  ^ 

C 


*0 

dj 

s- 


+->  cu 
c/>  E 
O)  '— 


>- 

oc 

< 


Q.  O) 
C>1  C 
ct; 


m  r—  al 
CQ 
CD  I 

CU  C 


CD  > 


< 

O' 


J-  >> 
U  -M 
in  -o 

Q  > 

+-*  Q 
CO  . 
UJ  CO 

-M  C 
*0  03 

*r-  (D 

^  s: 


h-  O  h-  •— I  t—  I— 


00»—  OOH-l~tOOOOOOt—  QCOOO'— lOCOH-  »—  O 
Dc:ooc_)oocooze3oooo<>jQ-»— t— zcjr-HrHz: 
cl.s:oo£;s:oooo(/^oos:q-cocos:s:odcos:s:oo 
tooccjcjc-<<t:ccj>oDCCocjci:<oo<c 

c/^  uo  oo  oo  CO  oo  CO  oo  oo  c/T  OD  on  CO  CO  oo  oo  oo  on  CO  CO 
«<<c<c<ccc<t<c<c<i:<a:<cc<cc<c<c<a:<c«3: 
cocococococococococococococococococococo 
‘a:c<cc<3:c<t:<c<:<<<cccc<c<<< 

^3:3:3:3:^^3C3:3:3::5::S::S:3::X3:3:^3: 

cocococococococococococococotocococococo 

■<<c<j:<3:<i^c<c<c«3:<5:<<:<i:c<cc«a:<cc 

cc:cDCci;cr;c3Cc3CQiQc:cr:Q;ci;cc:ct:DCcr:Dc:Dc:Q£a;c5c: 

<ccc<c<<tc<c<:<<c<<<c<<<: 

COCOCOCOCOCOCOCOCOCOCOCOCOCOCOCOCOCOCOCO 

«^cnLnLnLr)LnLO'^un«;r«srLn«:^'Lr)LnLr)^un«d-Ln 


CNjcvjrocvjt— (c^-r-ir-Hc^-unoromooor^t— ^’=^o 
CDC=>000OC300CT>000000Cr>0Cr>O 


cz>'^cocD>Lr>r-toOCT>mr^oOLr>CNJcoooco^«:3-c>Jco 
coc^lJn'=^cDcr3C^J»-^p>>.c^J^— ioLO«— icvjr-cf^oj.-HCNj 
r->.COr^(X3r^COr^r^CX)r^r^OOr^OOCOOOr^OOCOCO 

ooc:z>ooooooooooooooooo 

'=l-':3-c£3r^'=f-r^<::T>cr>i —  r^CTicjicaDcociTicooocncocD 
Lr)C\jLr)or>^.— ir^oocD^cDcx>Ln'^cncocD^LOcn^ 

OOOOOCDOOOOOOOOOOOOOC3 

co»— tcoc\jr«^r^coc\JtococococNJr^romcr>c7>m^ 

«— (1— itOCSJCOO^CVI.— «Ot— It— t^CDr— lOOOCM 

r— 4  1— H  mH  r*^  r— I  r*H  i-H  rH[  i"H 

OO0OOOO0C3OO0OOOOC5OCDO 

mOOr^CDOCDCOCDt— »CDrOCNIunOOC7)CDC\JLr)':3-':3- 

'^cooocX)cr>cr>cDOOCX)C30»-<oc7>cr)CDOoocor^ 

r-ioOOOOO«— 1»— JOOrH^OCZDOi— lOOO 


t^cror^CVJun*— ccvJCVJCOOOr-HOCOOCD'^CXJCD'^J- 
cococvJO'?}-^op^c£DC30r^r-HCOomoocDrN.io»-i 
»r~ICNJCNJCVJT— ICSJi— I.— Ii-Ht— ICNJ.— IC\JC\Ji— 1»— Ir-Hi-HCXI 

oooooooooooooooooooc? 

cyjcr)CO^councNjrocsJocoir>r"*cD'^Lr)o^mr- tcvj 
CNJcO'^CT>crjmLr)co*^<^cn<3-oo— ir^mooco 
cn'=i-^«:3-*=fLn^^Lr)'=P'?3-LnLr)LrjLr)«:i-LncoLO'=j- 


CTiCDm^r^UDCvJ«— i»?fCT>OCO’— •OOr^OUD'^fCO 
'=j-LncnLnLncnLnLncn':rcr)cncr)coiOLnLr)iomcn 


COOCno.— l^CT>C3^0COCO'— tCT>C\JCNJr-HCO*— <Or- 1 

cvjfOCsjroc^orocsJCvjrocNJCsjrocNJcocococNjrororo 

c^ooooooooooooooooooo 


cvimr-HPocvjmcvjmmmmcNjmLnco'cfcvJuncn^^i- 

LocnLocDLnLncnLnLOunLnLnuounLncncnuntnLr) 

ooc::>oo<3>ooooooooocD>ocDoo 


ocTicncT^oocococooor^f^r^p^cocDcocDcocnLn 
cDcocncniocnLnLncocnuoLncnLntnLncncnLnLr) 
r^r^r^t — .r^r^r^r^r^r^r^r^i — 


>~tc\Jco«=runcDr —  oocJiO^-^c^Jm'=^cncD^^ooo^o 


A-3 


1— 

}— 

h- 

h-  1— 

h— 

h— 

1 —  1 — 

o 

h- 

CXI 

h- 

h— 

rH 

h— 

CNJ  t— I 

CXI 

CXI  CNJ 

to 

y. 

y 

y 

y 

y 

y 

y  y 

y 

y 

y  y 

to 

y 

<_> 

CJ 

CD 

CD 

CD 

CD  CD 

CD 

CD 

CD  CD 

< 

CD 

00 

o 

CO 

o 

H- 

o  to 

h- 

1— 

1— 

to  1— 

1 — 

to 

to 

O 

CD  to 

0) 

1— 

!— 

CD 

CO 

z:  CD 

1— 

f-H 

to 

CD  to 

CNJ 

CD 

CO 

z 

CL  CD 

y 

y  c>o 

CO 

to 

y 

to  to 

y 

y 

y 

y 

to  y 

y  to 

to 

to 

CO  to 

JD 

<TJ 

<s> 

o 

c  <c  c  o 

<C  C  CD 

CD 

CD 

CD 

CCDCD«d:C<a:<C<a: 

oo 

oo 

oo 

CO 

to 

to 

to  to 

to 

to 

CO 

CO 

to  to 

to 

to 

to 

to 

to  to 

F 

<c 

< 

< 

<c 

<c 

c 

c  <c 

c 

<c 

< 

c 

c  <  c  c  <c  <c 

•<  < 

fd 

oo 

CO 

CO 

CO 

CO 

CO 

to  to 

CO 

to 

CO 

to 

to  to 

to 

to 

to 

to 

to  to 

.•> 

<c  <c  <t:  <c  •<  <c 

c  c  <:  < 

«d:  c 

<  <  <  < 

S- 

i«C 

rx: 

3X1 

ox:  :x: 

3x: 

ox 

NX 

3X 

3X  3X 

3X  3X 

NX 

3X 

3X  NX 

OX 

o 

IS 

IS 

3: 

^  3: 

^  IS 

IS 

+-» 

oo 

CO 

oo 

CO 

CO 

to 

to  to 

to 

to 

to  to 

to  to 

to 

to 

to 

to 

to  to 

to 

o 

<t  <  <t  <c  c 

<  c 

c 

<a:  <c  < 

c  <c  c  < 

<c  c  <c  <: 

<ai 

"O 

DC 

DC 

DC 

C3C 

DC  DC 

DC  DC 

nr 

rr 

cn 

or 

DC  DC  or  cor  or  or 

or  or 

or 

03 

<c  <c  <c  <  < 

c  <: 

< 

<. 

<  < 

<  < 

<  <  <  c 

<  c 

<dl 

J- 

oo 

oo 

(O 

to 

to 

CO 

to  to 

to 

to 

to 

to 

to  to 

to 

to 

to 

to 

to  to 

to 

Q_ 

<t:  <c  «=c  c  <c  < 

c  c  < 

<  <  < 

<  <  c  <t 

<:  <c 

<  <c 

<1. 

> 

LD 

uo 

uo 

uo 

uo 

U) 

uo  uo 

«d- 

uo 

LO 

•d- 

uo  uo 

•d- 

uo 

'd- 

uo 

^  «d- 

CNJ 

t/^ 

o 

x; 

-M 

O) 

O 

o 

03 

cn 

CD 

o  ^ 

m 

CNJ 

n- 

«d-  CNJ 

•d- 

cn 

cn 

cn 

{/) 

F 

O 

o 

C73 

o 

o 

o 

o  o 

(03 

o 

o 

C73 

o  o 

<03 

o 

o 

CD 

O  (33 

CD 

1— t 

1— ( 

1-H 

1-H 

h- 

h- 

OO 

« 

o 

J 

CH 

LU 

<D 

m 

CO 

uo 

uo 

CO 

uo 

CD  1—1  (30 

•d- 

<d- 

CnJ 

cn  (03 

tD 

CD 

C73 

OO  CNJ  1— 1 

CO 

(L_> 

03 

CXI 

uo  cs> 

1  CD 

CXI  CD 

«d-  CNJ 

cn 

’d- 

CNJ 

UO 

CNJ 

uo  r-^ 

<d- 

<c 

<_> 

F 

C 

00  oo 

CO  00  00  00 

r^C0<30C0COCO00r^ 

CD 

<D 

DC 

z: 

Li_ 

o 

o 

o 

o 

CD 

o 

CD  O 

o 

o 

CD 

CD 

O  CD 

O 

o 

o 

<D 

o  o 

O 

ct: 

1 

<c 

LU 

OJ 

c: 

oo 

C33 

(03 

CD 

'd- 

CD  (30 

CJ3 

^  CO 

r>. 

OO  tD 

C73 

cn  CO  00 

UO 

fO 

LO 

r*o 

o 

uo 

CD 

r'N. 

^  T— t 

03 

uo 

CXI 

UO 

(T3  o 

cn 

uo  00 

«d-  'd- 

cn 

s: 

OC 

n3 

a> 

«=J- 

tn 

'd"  cn 

'd-  «d- 

cn 

«d- 

»d- 

'd' 

cn  "d- 

'd' 

cn 

cn 

cn 

cn  m 

rH 

r> 

z: 

s:; 

« 

• 

• 

« 

• 

•  • 

• 

• 

« 

• 

• 

• 

•  • 

• 

oo 

LU 

C3 

o 

CD 

CD 

CD 

<D 

CD  O 

CD 

CD 

o 

CD 

o  o 

<D 

CD 

o 

o 

O  <D 

o 

LU 

Ll 

CD 

r«- 

oo 

r-H  CO  C^n 

«d- 

C73 

1-H 

cn 

tD  CXJ 

cn 

tD 

CXJ 

uo 

CXI  CTO 

uo 

1— 

Ll 

O- 

cn 

CD  CO 

1  CO  CNJ  C73  00 

o 

1— 1 

o 

O  CNJ 

1-H 

UO 

1 — 1 

CNJ 

1— t  O 

*d- 

< 

CO 

c: 

r— * 

^-1 

rH 

1— 1 

»— 1 

1-H 

•— 1  o 

t— < 

r-H  r*H 

t-H 

r-H 

rH 

rH  ^H 

o 

DC 

« 

« 

tt 

• 

* 

• 

■ 

«  « 

« 

* 

■  « 

t— 1 

31 

o 

o 

CD 

<D 

CD 

o 

o  o 

o 

o 

o 

o 

o  o 

o 

O 

o 

tD 

O  CD 

o 

Q 

1 

on 

CD 

C 

C\J  03  CO  Cvj  073  O 

^  CD 

uo 

uo 

CXI 

1-H  tD 

CD 

CD  cn 

UO  CNJ 

rH 

LU 

-M 

rd 

03  03  CO 

CD 

(03 

03 

O  CO  (30  CO  O  CO 

C73 

cn 

o 

tD 

O  <D 

x: 

(D 

o 

o 

CD 

<D 

o 

o  o 

1— 1 

o 

o 

o 

1-1  o 

o 

O 

1-H 

O 

1-H  1-H 

o 

>- 

Q_ 

s: 

• 

• 

• 

« 

• 

• 

■  * 

1— 

t-H 

r~~i 

f — ( 

1-H  «-H 

f-H 

1 — i 

l—\  1 - ( 

1-H 

1 — 1 

1-H  f-H 

C3 

Q 

o 

CD 

00  O  rn 

1— H 

03 

uo 

«d-  t— 1 

cn 

CO 

cn 

CD 

00  CNJ 

C73 

CNJ 

CNJ 

CD  'd' 

«d- 

-iC 

Cn 

o 

oo 

oo 

r-<  CO 

1-c  «d“ 

cn 

CO  CD 

CD  O 

OO  CNJ 

1 - 

CD 

oo  tD 

tD 

<C 

C£L 

o 

c 

CXI 

CXI 

CXI 

CXI 

CXI 

CXI  1—1 

CNJ 

1 — ( 

1— 1  CNJ 

CNJ 

CXJ 

1-H  1-H 

O 

> 

DC 

« 

C3 

CD 

o 

o 

o 

CD 

o 

o 

o  o 

o 

o 

o 

O 

o  o 

CD 

o 

o 

O 

o  o 

o 

t— 

i 

"^SZ. 

CD 

0) 

X 

CD 

p>. 

U3 

uo 

cn 

CNJ  uo 

cn 

C73 

1— t  CXI  ^d-  CO  CO  cn 

CNJ 

r-.  03 

(33 

C 

+-> 

<D 

o 

1-H 

CO 

'd- 

r>^ 

CT3 

CO  cn 

o 

CO  o 

^  C73 

O 

•d-  00 

UO 

o  m 

f— 1 

SI 

S3 

-o 

uo 

uo 

U3 

^  uo 

uo 

uo 

'd- 

uo 

UO  'd- 

UO 

«d- 

'd- 

uo  uo 

■d- 

3: 

c 

* 

• 

•  • 

« 

« 

«  « 

•  • 

CO 

I— H 

t— c 

1 

1—1 

1  ♦ 

f-H 

1-H 

^H 

1-H 

1— H  1-H 

1— I 

cc: 

t 

c_> 

X 

o 

C73 

00  oo 

r*- 

CD  tD 

CD 

uo 

CD 

'd* 

Id*  'd-  cn  cn 

^H 

CNJ 

1—1  <D 

cn 

oo 

O) 

0) 

CD 

CD 

uo 

uo 

U3 

uo 

uo  uo 

uo 

uo 

uo 

uo 

uo  uo 

uo 

uo 

uo 

UO 

UO  UO 

cn 

O 

-O 

1— H 

rH  1-H 

CD 

Q 

t- 

c: 

« 

• 

•  • 

• 

•  • 

CD 

>- 

CQ 

U 

>» 

CXI 

CXI 

f— 1 

t-l 

o 

o 

o  o 

o 

C73 

C73 

<33 

(33  CO 

O 

Q 

u 

+-> 

ro 

oo 

m 

cn 

cn 

m 

cn  cn 

cn 

cn 

cn 

cn 

cn  cn 

cn 

CNJ 

CNJ 

txi 

CNJ  CNJ 

O 

LU 

m 

■O 

o 

o 

o 

CD 

o 

o 

o  o 

o 

o 

o 

o 

o  o 

o 

O 

O 

O 

O  O 

o 

Q 

s> 

«=C 

QC 

uo 

uo 

•d*  CXJ 

»d-  CXJ 

"d-  cn 

uo 

cn 

uo 

cn  cn 

«d- 

cn 

CNJ 

cn  CNJ 

d* 

OO 

-M 

Q 

uo 

uo 

uo 

uo 

U3 

uo 

uo  uo 

uo 

uo 

uo 

uo 

uo  uo 

LO 

UO 

uo 

UO 

uo  uo 

d" 

s: 

CO 

o 

o 

o 

o 

CD 

o 

CD  CD 

o 

o 

o 

o 

o  o 

O 

o 

CD 

o 

O  CD 

CD 

o 

LU 

to 

• 

• 

• 

• 

• 

•  • 

* 

« 

• 

•  • 

• 

jZ! 

>> 

c 

C 

CD 

CD 

uo 

CD  CD 

uo  (30  'd-  CD  (03 

uo  (30  CT3 

uo  <03  CO  OO 

r».  tD 

(33 

-a 

td 

uo 

U3 

uo 

uo 

uo 

uo 

uo  uo 

uo 

uo 

uo 

UO 

uo  uo 

uo 

uo 

uo 

LT) 

UO  UO 

1— 1 

O' 

(D 

1 — 

r->» 

r-^ 

r-* 

r«^ 

r«- 

LU 

n3 

■y~ 

’ 

• 

* 

• 

* 

• 

•  • 

• 

• 

* 

* 

•  • 

* 

* 

• 

• 

* 

o 

> 

CNJ 

JX 

f-H 

CXJ 

m 

5 

6 

7 

8 
9 

o 

CNJ 

cn  ^ 

uo  tD 

r-^  00  (33  CD 

LU 

CL 

c 

1-H 

1-H 

rH 

1-H 

1-H 

1-H 

^H 

1— 1  CNJ 

to 

O 

fd 

c 

h- 

cc: 

CD 

A-4 


Technical /School  Knowledge:  Validities  of  Top  20  Equations  Ranked  According  to  Each  Performance  Index,  at 
the  74-104  Min.  Interval 


t 

> 

j 

1 

1— 

1 — 

t—  h- 

j— 

1— 

0 

1— 

1 

CM 

J— 

1— 

y—  1-1 

^  CM 

i-H 

CM  CM 

CM 

zz: 

CO 

1 

s: 

s: 

s: 

2: 

2:  ^ 

cn 

2: 

1 

CJ 

CD  0  CD  <->  CO 

CD  CD 

CD 

CD  <LD 

CD 

< 

CD 

(O 

1 

»— t 

0 

CO 

i—  0 

CO  0 

•— ( 

»— 

i —  1— 

CO  f— 

CO  f— 

0 

CO 

CO  CO 

(D 

1 

h-* 

h~ 

CD 

cn  3 

CD  3 

1— 

CO 

1—1  CO 

CD  .-1 

CD  CM 

CD 

CD-  CO 

1 

z:  cn 

CO  CO 

2: 

CO  2: 

CO  2; 

CO 

cn 

CO  CO 

JO 

<TJ 

1 

C_J 

CD  C  ■<  CD  C 

C«a:CDCDCDCD<tCD 

<  CD  <  <C  <  < 

1 

cn 

cn 

to  CO 

cn  cn 

CO  CO 

CO 

tn 

cn  cn 

CO  CO 

cn  CO 

cn 

cn 

CO  cn 

C- 

1 

<c 

<a:  <3:  <C  <C 

c<3:<3:ct<a:c<3;c<c<3: 

c  <  c  c 

03 

1 

cn 

00 

CO 

CO 

cn  cn 

CO  CO 

CO 

CO 

CO  CO 

CO  CO 

CO  CO 

CO 

CO 

CO  CO 

:> 

1 

<c  <a: 

<3:  <c  <c  <c 

<c<c<c<ccc<:<c 

<c  <c 

<c 

«3:  c  <3: 

u 

1 

V' 

X  X 

v* 

i»r  Dr 

v: 

DT  X  D*r 

0 

1 

:x: 

3: 

3 

3 

3  3 

3  3 

3 

3 

3  3 

3  3 

3  3 

3 

3 

3  3 

3 

4-» 

t 

cn 

cn 

CO 

CO 

CO  CO 

CO  CO 

CO 

CO 

CO  CO 

CO  CO 

CO  CO 

CO 

cn 

CO  CO 

cn 

0 

1 

<c 

c 

c  c 

c  <c 

<  < 

<  <c  c  < 

<  c 

<:  <  <c  c  c  <1: 

<c 

■n 

1 

cr 

or 

or 

nr 

or  or 

cr  nr 

(DT 

cd 

or  or 

IDT  or 

IDT  or 

or  or 

or  or 

nr 

<D 

1 

<c  <c 

<  C  C  < 

<  C  <  <  <t  < 

<  < 

<3:  <:  <  <  <:  <c 

<r 

$- 

1 

cn 

00 

00 

cn 

CO  CO 

CO  CO 

CO 

cn 

CO  CO 

00  CO 

CO  CO 

CO 

CO 

cn  cn 

cn 

Q_ 

1 

c  <c  <c  «o;  <c  < 

<  <  <  <c  <  <: 

<c  < 

<C  C  C  <C  <C  C 

<1: 

> 

1 

1 

cn 

cn 

cn 

cn 

cn  cn 

in  in 

cn 

cn  cn 

cn  ^ 

cn 

m 

*53'  *53- 

CM 

+-» 

(D 

i 

0 

0 

cn 

on 

r-H  0 

'!3-  0  on  CM 

1—1  CM 

'=3- 

on  ^ 

1 — 1 

1-H 

on  r>- 

on 

h: 

1 

0 

0 

cn 

0 

0  0 

0  0 

cn 

0 

CD  0 

0  cn 

0  cn 

0 

0 

0  cn 

n-. 

<u 

1 

T— H 

\ — \  t-H 

f-H  1— H 

1-H 

1— 

f— 

1 

1 

oo 

1 

o 

UJ 

1 

m 

00 

cn 

cn 

cn  m 

r-l  CD  00 

'=3- 

cn 

on  CM 

CO  CO  CO 

cn 

CM  1-H 

CO 

s: 

CD 

1 

CM 

cn 

0 

CO  t-K 

CD  CM 

on 

CM  '53- 

r«v  1—1 

cn  CM 

CM 

1-H 

cn  r--. 

*53 

CD 

c 

00  CO 

CO 

00 

cooor».oocooococo 

CO 

r- 

CD 

cr> 

F 

c 

1 

<D 

or 

1 

0 

0 

0 

0 

0  0 

0  0 

0 

0 

0  0 

0  0 

0  0 

0 

0 

0  0 

0 

oo 

U_ 

on 

UJ 

1 

1 

CO  CTi 

CD 

'=3'  CO 

CO  CO 

'=3- 

cn 

r^cocDOOcncncooo 

in 

o 

CD 

c 

1 

cn 

on 

cz> 

cn 

0".  CO 

cn 

CM 

in  0 

cn  in 

in  on 

CO 

*3-  *3- 

on 

oc 

or 

03 

1 

«:3-  on 

on  «:3' 

■cf  •53-  on 

'^3- 

':3- 

on  «:3‘ 

on  ^  on  on 

m  on 

C_J 

n3 

CD 

J 

c 

UJ 

x: 

2^ 

1 

0 

0 

0 

0 

0  CD 

CD  CD 

CD 

0 

0  0 

0  0 

0  CD 

0 

0 

CD  0 

0 

>- 

u_ 

cc 

1 

r««.. 

m 

00 

on 

1-H 

cn  CM 

CO  on 

CO  on 

cn 

CM 

CM  cn 

cn 

C 

u_ 

CD 

1 

CD  CO  CO  r-i 

cn  CM  00 

i-H 

0  CM 

0  0 

in  f— c 

CM 

1-H 

1-H  CD 

*3- 

Q. 

C7> 

J 

1— H 

1-H 

f— 1 

0  ^ 

.-H 

r— 4 

.—1  «— 1 

t-H  1-H 

1-H  1 — 1 

f-H 

0 

CO 

C 

i 

• 

ID 

•  r— 

or 

1 

0 

0 

0 

0 

0  0 

0  0 

0 

0 

0  0 

CD  0 

0  0 

0 

0 

0  CD 

CD 

C/O 

Q 

HZ 

V - .«• 

1 

t 

Cvj 

cr>  CO  CM  0  CTJ 

CO  "ct"  in  CM 

cn  CO 

r—t  rt 

0 

on 

CD 

CO  CM 

f — 1 

CD 

c 

c 

CTi 

(3  CO 

r— f 

cn  CO 

cn  CD  00  00  00 

0  00  on  cn 

CD 

0 

0  CD 

*3- 

UJ 

+-> 

1 

0 

0 

0 

r-H 

CD  0 

0  0 

t-l  0  0  CD 

f-H  0 

0  0 

0 

1-H 

1-H  1-H 

CD 

h- 

CU 

x: 

(D 

t 

■ 

9  • 

• 

•  • 

< 

3: 

s: 

t 

1 — 1 

1—1 

1 — 1 

ID 

t— 

0 

1 

CO  0 

on 

cn  cn 

t-H  ^ 

on 

on  CO  CM  CO  CO 

1^  cn  CM  CM 

0  *53' 

*53- 

oo 

CD 

1 

0 

m 

on 

^  00 

^  »--i 

on 

CO 

0 

CD  CO  CM  CO 

0 

CO  CD 

CD 

UJ 

or 

CD 

1 

CNJ 

CM 

CM 

CM 

CM  rU 

r-H  CM 

CM 

^  CM 

CM  1-H 

CM 

0 

0 

C 

1 

• 

• 

• 

•  • 

• 

«  • 

« 

.  « 

X 

CD 

or 

1 

0 

0 

0  0  CD  0 

CD  0 

0 

0 

0  0 

0  0 

000 

0 

0  0 

0 

UJ 

CO 

Q 

CO 

1 

1 

CD 

cn 

on  un 

in  CM 

«:3- 

cn 

on 

C\J  t-H 

cn  CO  CM 

m 

0^  cn 

cn 

<D 

c 

1 

0 

CO 

cn 

m  CO 

1-H 

CO 

0  cn 

'53-  0 

f=i-  CD 

m 

CO 

0  on 

f-H 

0 

+-> 

03 

I 

CO 

LO 

•=3- 

cn 

<3- 

m  ':3- 

in 

cn 

cn  un 

*3-  in 

*3- 

in  in 

*3- 

CD 

1 

• 

• 

00 

3= 

s: 

t 

w-^ 

1 

--I 

1  »-H 

r-H  •— H 

1 

i-H  i-H 

t-H  i-H 

1— H  r-H 

f-H 

r-H 

^—4  T— H 

t™4 

Q 

1 

CD 

X 

\ 

0 

CT>  CO  CO 

CO  CD 

CD 

CD 

cn 

’=3  '3- 

on  on 

CM 

«— 1 

f-H  CD 

on 

0 

cn 

0) 

1 

CO 

cn 

cn 

cn 

cn  in 

in  in 

cn 

cn 

cn  cn 

in  cn 

in  m 

in 

cn 

cn  cn 

on 

QC 

0 

-O 

1 

1-H 

t — 1 

f-H 

r-H 

CD 

CO 

J- 

c 

1 

• 

• 

• 

• 

CO 

1 

>~ 

CQ 

J- 

1 

CM 

CM 

i-H 

f-H 

t — ( 

1 - 1  T—\ 

« — 1 

CD 

t-h  0 

0  0 

cn  0 

cn  cn  cn  CO 

CD 

0 

0 

-M 

J 

cn 

on 

on 

on 

on  m 

on  on 

on 

m 

m  on 

on  on 

CM  on 

CM 

CM 

CM  CM 

0 

UJ 

to 

•0 

1 

0 

0 

0 

0 

0  0 

0  0  0  0  CD  CD 

0  0 

0  0 

0  CD  CD  CD 

CD 

1 

Q 

1 

<: 

a: 

00 

• 

• 

1 

cn 

CO 

CM 

CM 

on 

<e3- 

m 

cn  m 

on  in 

r-H  <3- 

CM 

on 

on  CM 

*=3- 

+-> 

Q 

1 

cn 

cn 

cn 

cn 

m  cn 

in  in 

in 

cn 

cn  in 

in  in 

m  m 

cn 

in 

cn  cn 

:z: 

0 

CO 

UJ 

CO 

1 

\ 

0 

0 

0 

cz> 

CD  0 

CD  CD 

0 

CD 

0  0 

0  0 

0  0 

0 

0 

0  CD 

0 

P 

1 

<C 

+-* 

c 

1 

CO 

CO 

cn 

00  CO 

CO  cn 

cn 

CO  cn  00  cn 

cn  in  CO  CO 

r>-  CD 

cn 

CD 

■0 

03 

1 

cn 

cn 

LO 

cn 

cn  in 

in  in 

cn 

in 

in  in 

cn  cn 

un  cn 

in 

in 

in  in 

O' 

CD 

) 

r>^ 

UJ 

na 

s: 

1 

1 

• 

• 

• 

•  • 

•  • 

* 

• 

0 

> 

C\J 

JX 

1 

f-H 

CM 

on 

5 

6 

7 

8 
9 

0 

r-i  CM 

on  ^ 

cn  CO 

CO  cn  0 

UJ 

CL. 

d 

J 

^H 

1-H  1-H 

1-H  1-H 

f-H 

f-H  CM 

cn 

0 

03 

1 

< 

1— 

or 

1 

CQ 

A-5 


X 

■o 


Q 

u 

c 


a. 

CT> 

-n 

t 

1 

nr 

nr 

nr 

nr 

nr 

nr 

nr 

nr 

nr 

nr 

nr 

nr 

nr 

nr 

nr 

cx: 

or 

or 

nr 

nr 

1 

1 

nr 

JZ 

0 

1 

<ar 

< 

<c 

< 

<■ 

c 

<t: 

< 

< 

<c 

< 

c 

< 

< 

<c 

0: 

c 

0: 

0: 

1 

0: 

u 

CO 

S- 

1 

t/o 

00 

CO 

00 

CO 

CO 

(jTi 

CO 

CO 

CO 

CO 

CO 

CO 

CO 

CO 

CO 

CO 

CO 

CO 

CO 

1 

CO 

0 

00 

ci_ 

1 

< 

<c 

<c 

< 

< 

< 

<c 

<c 

<C 

< 

< 

< 

< 

< 

c 

c 

<c 

< 

0: 

I 

c 

LU 

0 

QZ 

> 

1 

LD 

CD 

0- 

CD 

cn 

0- 

CD 

CD 

uo 

CD 

0- 

CD 

CD 

CD 

ID 

CD 

ro 

CD 

CD 

CD 

1 

CNJ 

0 

CD 

1 

i 

+J 

<c 

4-> 

CT 

>- 

CO 

0 

1 

rO 

cr> 

0- 

CO 

0- 

CD 

ID 

ro 

CD 

0- 

CSJ 

ro 

CD 

0 

0 

t — 1 

0- 

C\J 

1 

ro 

C 

cc 

0 

e 

1 

CT) 

0 

CO 

0 

0 

00 

0 

CT> 

cr> 

0 

cn 

OO 

0 

0 

0 

0 

0 

CJ> 

0 

1 

<c 

1 — 

1 

’—I 

T  ■< 

,— ( 

f-H 

I— ' 

1 

-o 

o 

o 

■g 


+J 

03 

3 

cr 


O 

w 

0 

■M 

T3 


0 

o> 

"S  "fl 
> 
X  (- 
O  0 
C  4J 
^  2mS  C 

•a  •— ( 


< 

0 


1 

U 


z:  u_  a: 

I 

UJ  0 


CL  0 
CO  CO 
—  C 
□Z  QZ 


CD  CD  Qc: 

i 

CQ  0 

4_J  c 

CD  x:  0 
3:  0 
oo 


1 


o 

LU 


< 

O' 


*  X 

O)  0 

o  *a 
S-  c 

CQ  t— • 


t.  >» 
O  4-> 
CO  TD 

Q  ^ 

+-»  o 

lO 

LU  CO 
>> 

+->  c 
-O  0 


0 


to  CO 

z:  ^ 

CJ  CD 


to  to 
s:  s: 

O  CD 


O  CD  O  CD  CD 


O  O  O  O  CD  h-  O  O  O  h-  O  I—  O  O  O  )—  I—  O 
ZZZZQ_tOZZZCOQ_tOZ2Z:z:tO  totoz 

to  to  to  to  to  s:  to  CO  to  s:  to  s;  to  to  to  s  z:  s:  to 
<;<i:c<ccocccoccD<3:<cccD  cdc-dc 

totototoootototooootocotoooooto 

CDCDCDCDZZC->CDCDZ:Z:ZCDCDCDZ=Z:ZZZ:CD 

totototototototototototocotototocototoco 
<C  <C  <C  <C  *3^  *3!!i  <C 

313:3:3:3:3:3:3:3:3:3:3:3:3:3:3:3:3:3:3: 

totototocotototototototocototototoiototo 


oc2Doooun<:foocnroT-HC3^«;j-Lncvj.— tcOLOcncr* 
r-HCTi,— (COtOt— iOOCOCOtDT-HOOr»^COcn»-«COCOCX> 
CMCVJC\J0-OJCSJLn0-O>JO«JCVI0''^t-0-'=1-'d”CsJ'cr0‘0- 

000000C3C3000000000000 

to»— «CNjrHCvJC\j0-c\j'=roocr>tooc\JCNjir)r^«— ir^ 
t£3OOr^Lntocn0-r>.cn^r^cor^cOtoo^ocT>tX3to 

r-(r-<r-HCNJ»-tT-<CMCSJ^CNJ,-4CVJC\JCVJCVJCNjCSJC\JCSJC\J 

OOOOOOOOOOCDOOOOOOOOO 


00000000000000000000 

Oi— tcvJcDoroi— 1«— (cncnooO’— •cnr^tn^'CNjror^ 
nLn0-0-r^r^Lr>Lntococor^tounLn<X)oococovo 

OOOOCTiOOOO'iCD'iOOOOOCTiOOCTiCn 

OOOOOOOOOOOOOOOOOOOO 

^r^C\J«— (tOCVJ.— (0»^OCX5«— <OtX)OCDtDC\J0-CO 
oro^cor-^cr>0'Cnunmcor^0’0‘ijni— (orofo.— t 
rnCSiCOCNJCMCSICMCVJCvJCVjCVJCsJCNJCvJCvJOvirnCvJCNJCNi 

OOOOOOOOOOOOOOOOOOOO 

tnLO<D>oror^otor^CNjmm0"^r^cr»»— 

ooc7)cr>oooo»-HT--iCMCNjcsJC\JCNJC\JCsjrocomro 

CNjCNJcvjrororomrorommnmmmmrommro 


a>ocDr>..— icnro«:t'r-.r^cr)LntDcvjcT>co»-H0*t— 10 
r^or^ocOr^oooor^cD'joooor^oo^— I 

Oi— (0000»— <00000»— <«-HO’— <0»-Hr— (»— I 


H  CD  o  ro  cvj  1 


<  m  CD  cvj  cvj  0- 


tDi— tCDOr-*tD(>JOO' 

0»— tOi~HOO«— (i— 1«— !•— 'O*— 1^— (»— 1»— i^Oi— «t— 1«— I 
OOOOOOOOOOOOOOOOOOOO 


OCT)OOCNl»*-4Cr^r-HOOCvJr-(.— (OC7>t— (CNJOOCTJ 
^rO0-0*«=J‘0"CO0-0-0-0-0-0-0-m0-0'0-0-rO 
OOOOOOOOOOOOOOOOOOOO 


cDOOcno0-o»— «r^cnrooLncTiC3^cocDuotDcDcr> 
(Dmm'=t-rom0‘rommrocorommrocsjromro 
r^r^r^r^p^r'vr^r^p^r^r^r^r'vr^r^r^r^r^t^r^ 


T— (Osjm0'LntDr^<X)CT>o< 


<c\jrio0-LncDr^cOCT>o 


A-6 


X 

Q) 


C3 

U 

ez 


o 

D> 


■o 

U 

o 

u 


■g 


3 

cr 

LU 


CL 

o 


"p 

a 


a 

cn 

^  a 
Ji 

5  a; 


6  c 

jC  ♦»“ 
(J  £ 
LT) 

<—  O 
fC  ^ 

u  t 


QC 

<-> 

o: 


:3  -M  fXJ 

^  cu 

00:2:2: 


•  x: 
O)  CD 
o  -a 


CD 

O 


< 

O' 


$— 

o  +-» 
c/)  *0 


■  I J—  J—  I — «  »— I  Qt  t— I 

(OOOOr-HCvJH-h-OI— 


OOOOOOOOOOCJ) 


c_>  o  o  o 


0000000000001— h-Oh-  t 
:;zzzzscz:2c:2:2=;2czzooooci_co{ 

OOCOOOOOOOOOOOt/^OOtyOOOOOSSOOS: 

c<<c<<<<<:<cc<oo<o< 


00  ^ 
2:  2: 

C-J  <-J 


0000 


00C/500  00  00  00  00  00  00  00  00  00OOOO 

0000000000002:2:3:2:^^:^-^-^ 
oooooooooooooooooooooooooooooocooooooooo 

2:3:3:3:3:3:3:3:3:3:3:3:3:3:3:3:3::2^^ 

oooooooooooooooocy^ooooooooooooooooc/^oooo 

<c<t:<cc<<<<<c<:<<'<c<c<c<a:<i:<c 

c2c2c2c^c2D^c2c2c2c^c2c2c2aCc2cCcciccicx:ai 
<c<t:<a:<t:<cc<<c<:<tc<<i:<:<:<<<c< 
00  00  00  00  00  00  00  00  00  00  00  00  00  00  00(>0  00  00  00  00 
c<c<t:<c<cc«a:cccc<c<<<<t:<c<<a:<cc 

LO«^to»-nvooouotDa^i-ntD«:i‘LDLntDuomtD*^ 


oooco«^cOLOCNj^i»nr^CT>uno>OLncr>0'r-(CT> 

T-(»-Hroooor^<X)roco«^o»-H,— icococX).~<Lnoo 

000000C30000000000000 

ocvj»— •oJ'^^ocovD'^fojr^csJocxJr^^uooj^ 
or^Lnr^«::a-cocovDr».cr>otDcr>cotoocooc3^cva 
li— ICnJC\JC\Ji-HCvJCvJCNJ»— ICVJCNJf— IC>Ji— ICVJOJCVICNjm 

OOOOOOOOOOOOOOOOOOOCD 


00000000000000000000 

oc\JtOi— 1»— 1»— (Lor^i— tmunr^rocoocNim^uncTJ 
ro^^ir>u:iLnuoLn<x>ooLDr^r^r>.cococc>cooo 

CTi  Cr>  O  CT>  C7»  CT>  O  CTi  CTl  CT)  O  O  CTi  CT>  O  O  O  O  O  O 

00000000000000000000 
.-HCvjr— to^r^oooo'^fi.orocsjT— «ocvj«:3‘vooun 

o«-HcoCTi'=J-m«:j'Ln’^Lncr>r-icTir^r^mroo<— tco 

mrocNjCNJCNJOOCNJcvjcNjcNJCNjcvJCSicsicvJCNjoorocsJCNj 

00000000000000000000 

t-nooooiD^p^^r^LD^r^roro.— icNjt— icr>f^ 
cOC7»Of-<ocTJC\JCvjCNj»-«romoc\Jocoromc\j'=i- 
csjcvjrommcvjmmrororDfommmrorocomm 


oor^^mocNJcTiOr^»MOunLnt-H<:d‘»-H,-Hcoo 
r^r^a>C7iooooocT)0»— (f^ocooor'-oo 
OOOOt-Hr-lf-HOt-IOO»-HOOOT-H,-HO»*HO 


oooocvj.— icvi^moc3^«=J-tDor^CNjcviorocn 

<~4  T— 4  CJ  f  ^  t-H  1—4  1-H  C5  i”H  C3 

00000000000000000000 


ooo»— (CT>cr>ocD.-HOOcr>»~4i— icvjoocvjT-4pn 
ro  m  m  ^  m 

00000000000000000000 


tDrOOr^i— 4COOCOC7>LnOCTiOLn'«:t-oOLr)OT— 1 

rom'=rn«:i-mmmcr>ro^mrororororocvjro(r> 
—  r^r^p^r«..r^r^r>..r^r^r'^r^r««.i — 


1— icvjro»=t*LntDr^coooi 


4CvjcD^Lntor^cocr)0 


■o 

CD 


A-7 


o 

5  C 

c^ 

^  •<“ 

U  X 

to 

• 

»—  o 

t— ( 

ro  »— < 

< 

U  I 
•1-  ^ 

CD 

c 

U  CD 

fO 

c3  sz 

+-»  cu 

00  E 

CD  -r- 


<_>  •  CD 

E  C 
z:  CD  cr: 


o 

ac 

e? 


Q)  c: 

+->  cd 

a_  jz  a> 
3  ^ 


o 

CD 

cc:  ^  cn 
CD  c: 
cj?  Qc: 

CT) 

CQ  I 


CQ 

Q 


< 

czr 


CD  c: 
n3 
CD 


•  X 

cn  CD 

o  -a 

j-  c 
CD  •— » 

I— 

o  +-> 
to  "O 

Q  > 

Q 
to  • 

UJ  t/O 

+->  c 
-O 


CD 


03 


C_)  t—  C>0  C/^ 

o-  C/0  oc:  q: 
c/o  s:  a,  o- 
<C  c_>  to  c>n 


H*  O  to  h- 
CO  to  <j:  Dc:  to 
s:  x;  a.  c3l  s: 
c_)  c_)  to  to  o 


COC^<LDtOtD>tOtOtOCD>(0(OOI—  tOtOi*fCOOC_><0 
CJCL.Q_CDQ-<_><_>DC;a-CDC_><CODC:DC:^C_3<Q_QC: 

tO(otototocotoci,tototocx.s:a-Q-toton-(OQ- 

<a:<cc<t:c<t:<a:to<cc«t:tocjtotoc«a:to<t:<o 

tototototototocotototocototococococototo 

<C<C<<C<C<CC<:<<<<<;<C<< 

aiacc3cc2i:cx:c:2cccc0cc2ccc£c2c2ccc2ccccc2cc 

<<i:<3:<<3;«^<cc<<ccc<i:c<;c<<t<:<a; 

totocototocotocototototototototototocoto 

c<c<<c<<c<c<a:<<c<<<<cc<c<c<<<3: 

^^•d-^m<d-ro^tn^Ln^rOLnmroLnro^^ 


crir^«d-00C30.-Hmc£3r0'^«d-c\Jt-HCvJ00or^<:3-tn'd- 

CT>ooocj)CX)CT>cocnoooocx)ocooa^c3ocDcr> 


cnto«d-'4-or^r^C3or^^cx>c\JLntnncsitototOT— I 
ooocrjLOOtntnmT—iLnr^'d-^tD'^l-uooorocoro 
r-(r-«0»— iCNiOOi— lOOCVJO^*— I 

OOOOOOOOOOCZSOOcdcDOOOOCO 

^'d-tncD(X)cOLncocoorou->Lnr>.cx3CNjmcDOO'=J- 
oooococnooaooooi— It— ii—(.—(c\jCNJCNJCNjro 

C  ^  i-H  1-H  i-H  1-H  i-H  1-H  t-H  i-H  i— H  i-H 

ooooooocDooooiooooczjooo 

O^rOr^CDCNJ.— iCD^^-OCTJCOt— iC0CDCDOCD«d-'^ 
tDtn'=:j''d''d‘LOtntni— itDCTitotn*— I'd'tOi-HtDCDoo^ 
OCIOCDOOOOOt— tOOOO«-HOOt— (OOO 

C30000000000000000000 

O’d'CvjuootDcsjtDt— itomroLDj-HtoorotDtntn 
uO’d'T— (otnro'd'crjtDrot— iromoocvJtnror^i— I 
OO^.— lOOOOOO^OO-*<rHOOOO^ 


tncoL^>CT>Ln^otntDmrv.r^^ocDLOC^JtX)^^o*^^c\l 

CT)r^tocT>tDOOP^CNJOOcocor^cvjco^taDtncT>oo 

O0000^.-I0.-I0000.-H0»-H0000 

oooooooooooooooooooo 

councNjcsioocvjcsjt— loa^|^»~tun'^^^c^Jcou^aoT--^ 
cvJOtntDt— 1»— ifO«d‘CNjr^cDCDO^touDCNjr^roto 


toto«— icx)*— fc\Jcr>mr^nmoc\j*:d-C3Dtno^cvJCT^ 
tDuncotOLr)coLr)tr)COr^aoto*=:J-cx)’=l'tDC7>uDCX>r^ 
OOOOOCD>Oe3(=>OOOOOOC3C5CDOO 


^OJm^CsJCOmcsJCOtOCTirOT— tr^CNI'd-CDror^cO 

oooooooooez>oooooooooc:> 

OOOOOOOOOOOOOOOOOOOCZ) 


t^csJCNJCsJCsiroo'^rOr— cmcvi*— <’^■0000 
OOOOOOOOOOOOOOOOOOOO 


«^coror— im^«— ic\JO>coroc:Ocn<=rotDr^r^tocvj 
CNjcNimncNjcNJCvimcsicotomCNjmmcvJOJcocvjro 
—  r^r^r^r^r^r>.r'^r^r^r^r^r^r^r^r^ 


A-8 


<u  c 
a: «—  (O 
erj  a> 


or  CD 
u  c 

O  —  QC 


CJ<_>C_JOOO<_^CJOCJCJ 


oo  oo  »— 

x:  s:  z 

o  o  o 

»—OOI—»-^J— 
CM  c  c  OO  CO  »-• 

s:  Q_  Q.  z  ac  z: 

O  OO  C>0  O  C->  C-> 


»—  I—  >— O*— 'I—  •— 'I—  t—OI— H-Ot— <->000*— '»— 
COvOOO«t>—  OOH-  OOCO<t:t/OCO<3:cOQ-Z<C<Ctr^ 
z:aE:z:Q-s:s:2:ac^Q_3:3:a.z:oocooua-s:z 
cjocjcocjcjc-joocooococjCCcococjcj 

ooocJoooooooocjoocooooo 

<:<c<Q-<:<<cc<a;a-c<Q-<3=cj3=c2:c<: 

a-Q.a.OOQ.Q.Q-a.Q-COQ-Q-COQ.OOCOCOt>Oa.Q. 

cnootoCt/itooooocoCooooCco<CCCcoco 

aEaEs:e3a.Q-Q.s:Q-ozQ_3:s:oujztuae:E 

OOC/IOOCOCOCOOOCOCOCOt/lCOCOC/^OOCOCOCOCOCO 

cncOOOi— TiiToOi^*— <COCO'^i«£  •— ><— <cocococococo 

OC3<J3UJ2:<J3Z:LUOC_>UJ^LULiJC_JOC3C-)tSC3 

cococooocococococococotocntocotocyocoooco 

cc<c<c<:<c<ccc<c<c<c<:<<<:c<<cc 

OOt/lOOOOCOCOCOCOCOOOOOCOCOCOCOCOCOOOCOCO 

cooococooncococococooococococooocooooot/o 

cc<c<j:<<ccc<:<<c<:<<c<ccc<c 

cooocoooont/ooococ/^cooocococococococococ/) 
^  ^  ^  ^  ^  ^  ^  ^  ^  ^ 

§  §  §  §  §  §  g  §  §  §  §  S  SE  §  §  §  g  S  §  g 


(X)COOOCT>COCT>COCOCr>C3^COr>-.r^OOa^OOOOOOr>. 


CMCOr^cn»-HCM*:r{T»oocooor-,r-(CMCo*-^oocnr^ 

rv.mLnror^CT>COCTtoro<£>ror^co<^m^o^^»-H 

r^COCOOOOOCOCOr^O>COCOr>.r^COi^OOr^cX3COr^ 

ooooooooo<z>oooooooooo 

CNJCT>WDO»=i“nOUDOcDCMCOCOCT>CMCM';i“ir>LnC7i 
rouDuOi— <Lnm«^^CMooooocM<£3ocMcn^coco 
rocorococnfnmmmcMmmmmmmmmmm 


oooooooooooooooooooo 


•com«-HCMmo><r'=r«— 'vooco<^^»— 'cocz>»^ 
^«:r^ooo^^DO•^r^^^«:^■^o«5r«;rcD*c^-«c^»-^c^i«;r 

CM.— t-^T-Hr-^»-ICMCMr-(--4»-i^^^^CMi— iCM'^r-l 


000<=>000c=>0c=>0000<=>c30c=>0c3 


*:}-^ir)or«^mor^^cMioi^mcocNioocMCOi-nvo 

ior>.r^cr><^cor^«r>coo^cor^oov£3P^ir)cO<S'cDr>. 

oooooooooooooooooooo 


^cocMcoo%mLn^CMCT>«:rr>.mtom^<rtoooco 
.-H^j-rN-ncDm.— i.— (iD.“«^ir)^yDmo^ooco^ 

mCMCMCMCMCMrOmtNJCNJCMCMCMCMCMnCMCMCMCM 


oooooooooooooooooooo 


oooo.^r«vCMT-ima->o.-tmocMOOcr>CM^o 

«3-Lnocr»r^cT>coinor-iior^coor^r««.oooo 


^^rommmmunmpomm.— tCMcor-Hm.— t^uo— t 
mmmmrorornrocnrnmmmmmrncommro 
oooooooooooooooooooo 


oooor-ico.— laoo.— <oocDoo.^ocr>ocTvcj» 
'4-^^u^«:i*Ln^«:fur3LnLn':runtntr)Ln^^«:r'^ 
oooooooooooooooooooo 


tr)^«sr^.:;j-.=rrr^<3“mroronnfncocommm 
r^r^r^p>.r^r^r^r««.r^r^p^p^r^r^r«^r^r^r^f — 


.-HCMm-'ct-unor'N.oocriO— ^CMm^iDo^oooo 


A-9 


K 

'I 

c; 


0) 

Q. 


< 

a 

-Q 


o 

u 

u 


•S 


10 

3 


CM 

O. 

o 


(A 

O 


y 

15 


a> 

cn*— 

■8  S 

Is 


8-5 


rO  I 
U  ^ 

cn 

c  ^ 

85 


T3 

(U 

X- 


>- 

QC 


O;  C 
QC  <—  ft3 
fT3  QJ 


>- 

I— 

O 


es  — 
CQ 
CQ  I 
<D 


OQ 

Q 

LiJ 


C 

ID 

O' 


o 

o 

I- 

m 


s-  >> 

U  -M 
t/1  *o 

Q  > 


>s 

+-> 

“O  c 


o  o  o  o  o  o  o 


r-i  CM 

<->  o 


CM 

O 


CM  t— «  r-4  «--•  CM  CM 

^  z:  z:  z:  z:  z: 

<_>  CJ  O  CJ  CJ  CJ 


t-JtJOC_>OOCJOOOCJOOC_3<LJCJC_)C_>C_>0 

»— OC-JO>—  »—  *1— 't-Ti-ri— “cj 

o^Q_Q_zooooz:^—o^H-^— t— ooooooonH-h—onci. 
z<^cnc^z:z:uo:Ezsz:z:z:z:zz:z:^z:<>o 

C_J<:<t:<CC_>0<C-)<_)<_3C_>CJtJC^CJ00C_3C_;C 

O  o  c_j  o'  c/0  c_)  o  c>3  cj*  <_r  o*  o  t-}  H-* ili  o' 
a-zzz:a.Q-OQ.z:Q-Q.Q.ci-:2:zcLoooo3Ez 
C^c0(>000(/)00u0c0c/0ty000cy00ncy000{/^acz:cn(/l 
<c<<c<t<:<:<:<s:<:<<<<<c<:coo<:< 

zac^o3:3i:Lu^cjz:z:z:sz:z:z:ci.ci.c3:E 

ooooc/ioocoooooooonoooocoooooooooc/^oococo 

c*a:c<cccc«t:«a:ccc<cc<c<cccc<3: 

I— It— (onoo«— roooooo*— •:^ic:oo>— ^ 
ujcj3Ljjujz:z:oLiJujocsujoc3caujEz;ouj 
cooocot/ioooococ/7oocoooc>ot/)ooc/ic/5c/iooooco 
cc<tccc<c<j:<c<;cc<<:c<:ccc 

ooooooc/1c/^ooooooci<ouocoooc/^c/^oooo^/^c/^cou^ 

<t<3:c<<c<c<:«a:c«a:<t:<c<c<tc<<c«<<cc 

ooooco(>ooot>oooooooooononc>ococ/ion(>oooonoo 

tot>oc/>ooontnc/ioooocooot/^oo(joooiyocooot/^co 

ccc<c<c<c<c<<ccc<ccc<:<c«a:<c 

§  g  S  g  §  5  §  ?  S  g  5  §  §  g  ?  ?  ?  g  g  SE 

^^c^^^ooc/^(/^oo^>oo^^ooncoo^o^c/^o^o^c/)^>^co 

<t:<:<<j:<:<:<<c<j:<:<c<<cc-<<:<c 

CJ»C7>010C7^00COCT^CT^OOC7>0^000>CnOOCT>CT» 


rrjm»~ior>-.crmLn»-H^r^cM»— tcMir>mco«^cD— ( 
coocownunoouDootncocDtouncoiDi-nLDvo 


CM»-HCMO^oicocMCM.~HCMcncj^ococnr»«-r^como 

CJ>coCO«^r«v.cOCOr^r^(>n^r^CT>vDuno^p>>-.cor^CO 

CX5COCOCOOOCOCOOOCOOOOOCOCOCOCOCOC30COOOCO 

OOOOOOOOOOOOOOOOOOCDO 

cnT-Hrotr3m^«^CJicocococT>oocMCMocMr-H^r«s. 
0«-HCM*^00CMCMCM«-H.— iCMr^t— I.— iCT>CTiOOO 

OOOOOOOOOOOOOOOOOCDOO 

»-Hro^OO»^o*— ti— i^OLDr-HCMr^^unoOOr^cn 

or'.oor^coo^oocr»cr>oooocr>C7^cooO'^ocT»cocT» 

CM»— l«-H»-H,— (r— li^f— It-H.— It-H,— 1»— iCMi— <•— Ir-it— • 

OOOOOOOCDOOOOOOOOOOOO 

oocOLOr^f^*— (CMCO«^romc7>r^r^r^i-H^^C7^o 
r^O0>Grjf^r^OC0f^0^OC0r^r^r^C0r^r^r*^O 
0000000000000000000—1 


r^COrOCMCMOCMOCTif^— itncD— •CMLTJmr^uOO 
OCDrv.r«^OiOir)OOCt)COOOCX30000— <— lOr^CTJ 
CMCMCMCMCMrOCMCMCMCMCMCMCMcnCMcnmcnCMCM 

oooooooooooooooooooo 

mooocsjm^mcocX)^mo>cncT^cor^CT»co«J*^ 
muoLDLomrocO':?'— i^^’^'^LDCMCMcomm^t'co 


^'^'^s-mromncMmcMCMCMmmmcMCMCMmcM 

r^r^r^r>.r>.rv.r^r^r^r>..r*«.f^r«vrN.f>vr»s.r«>r^r«s.r^ 


0<-OcOVDCOOCOOOCOCOtOOCDcOOCOOCOO 

rnrornrncncnrnrncncornrncncnrnrncnrncofy^ 

oooooooooooooooooooo 


OOO— «OOCOOOCJ>OCTJOf'^CT>CT>CJ>COOOOOO 
OOOOOOOOOOOOOOOOOOOO 


^  o<ir>coooocn— loocT^— 'OOCMoo— <  o 

CO  CO  r>^  cO  cO  p*^  <jO 

^  r>.  p^  p««  p^  p*..  p>s.  p^  p^  p^  pn.  p^. 


— *CMm^cocor>.00CTsO' 


A-10 


H- 

h— 

1 

1 

CM 

CM 

CM 

1 

£ 

2 

2 

2 

t 

CD 

CJ 

CD 

CJ 

1 

1—  t— 

»—  1— 

h-  h-^ 

h- 

h-  •— 1 1— 

y_ 

h- 

I— 

t—  1— 

H-  1— 

1 

f-H  t—  1—1 

f-H  CM 

CM  CM 

1 

^  i: 

2:  2: 

2:  s: 

2: 

2  2  2 

2  2 

2  2  2 

2 

2  2 

2  2 

1 

O  C_5 

c_i  0 

CD  CD 

CD 

cj  0  0 

0  CJ 

CJ  CJ 

CJ 

CJ 

CJ  CJ 

0  CD 

1 

w  u 

|_H 

t-H  H- 

f-H  *-H 

1— t  l-H 

•-H 

l-H 

f-f  h-' 

l-H  h— 

1 

h—  H- 

H-  h- 

t—  1— 

h- 

)—  CO  1— 

1—  t— 

1— 

h- 

1— 

)—  ^ 

l-H 

1 

2:  2: 

2:  2: 

2:  2:  2: 

2  2  2  2  2 

22222222 

1 

0000 

C->  c_>  0 

0 

CJ  CJ 

CJ  0 

CD 

CJ 

CJ  CJ 

CJ  CJ 

1 

»—  0  0  0 

>»Ch-h— h-OI—  1— I— 

h-  h- 

h-' 

h-' 

or  l-H 

^  l-H 

1 

on  Q- 

Z 

Z  C/O  CO  CO  Z  CO 

CO  CO 

to  CO  to  to  0  H- 

CO 

1 

2:tyooo(-nt/>2:2:2:oo2: 

2  2 

22220.222 

1 

«_5  <a: 

<  < 

C  CD  CD  CD  <C  CD 

CJ  CJ 

CJ  CJ 

CJ 

CJ 

CO  CJ 

CJ  CJ 

1 

0  0  0  iti 

cn  CD 

CJ 

0  0  0 

CD  >*: 

cx:  0  CJ  CJ 

CJ  CJ 

1 

Q_  z 

Z  JE. 

CD  CL. 

O- 

2  2  0- 

z  z 

z  2  0 

z 

CX.  Q- 

Cl.  cc 

1 

00  on 

CO  CO 

COCOCOCOCOCOCOC/1 

to  CO 

d. 

CO 

CO  to 

to  to 

1 

<  <c 

<:  < 

c<:<c<cc<< 

<<ccoc<a:<<< 

1 

^  CO 

0  0  CO  *-• 

DC  CO  0 

^  0  ^ 

DC  ^ 

\ 

^  2: 

2:  0 

UJ  z 

Z  to  LU  2  2  2 

2  CD  CL 

2 

Z  2 

2  2 

1 

00  CO 

CO  tyO 

CO  CO  to  CO  CO  CO 

CO  CO 

CO  CO  CO  CO 

to  CO  to  to 

1 

c  <  <  < 

<  <t 

< 

<  <  C 

<  < 

<:  < 

<c 

<  <  c 

<  < 

1 

1— 1  •— 1 

CO  1— > 

CO 

cncn 

CO  cn 

t-H  c>n 

f-H 

Id  l-H 

l-H  to 

1 

LU  UJ 

0  LU 

tj  ac 

2 

CU  CD  CD 

CD  CD 

LU  CJ 

2 

LU 

2  UJ 

LU  CD 

1 

00  00  00  C/^ 

CO  CO 

CO 

cn  cn  cn 

CO  to 

to  CO 

CO 

to 

to  CO 

CO  to 

1 

<c 

<t  <C 

<t  <c 

<<<:<: 

<c  < 

cc  <c  <c 

c  <: 

c  <c 

1 

(yO  00 

CO  CO 

to  CO 

CO 

CO  CO  CO 

CO  CO 

to  CO 

to 

CO 

to  CO 

CO  CO 

1 

<c  c  c  < 

<  < 

<c 

<c  <  <t 

<  <c 

c  < 

< 

< 

c  <: 

<  c 

1 

00  00 

CO  CO 

CO  CO 

<0 

to  to  to 

cn  cn 

CO  CO 

CO 

to 

to  to 

cn  to 

1 

c  c  c  <: 

<  < 

< 

c  <:  c 

c  c 

<2:  < 

<c 

<t  <  c 

c  <c 

1 

iC 

ac  DC 

DC 

DC  ^  Zk:: 

DC  DC 

1 

::d 

3:  IE 

3: 

3:  3:  3: 

3  3 

3  3 

3 

3 

3  3 

3  3 

1 

3 

CO  CO 

CO  CO 

CO  CO 

to 

to  CO  CO 

to  CO 

CO  CO 

to 

CO 

CO  to 

CO  CO 

1 

CO 

<c  c  <c  c 

<  c  c 

<c  t:  c 

c  <c 

<c  < 

c 

<c 

c  c 

c  <c 

1 

c 

QO  q: 

CC  QC 

DC  QC 

ct: 

QC  or  Qg 

a:  a: 

oc  q:  cc 

QC  CJCOc 

CC  CC. 

1 

CC 

<:  c  c  <c 

c  <: 

< 

^ 

<  c 

<  < 

< 

c 

c  c 

<  c 

1 

<3: 

CO  CO 

CO  CO 

to  CO 

to 

CO  CO  CO 

to  cn 

to  to 

to 

CO 

to  CO 

CO  CO 

1 

CO 

<C  < 

<c  < 

<  <c 

c 

<:  c  c 

c  c 

<  < 

< 

<  <  c 

<  < 

<: 

cr>  01  cn  CT» 

0 

cr»  cr>  0 

0  CTi 

0  tn  cn 

cn 

cn  <n 

cn  cn 

1 

CM 

r*H 

1  f 

m  r-H 

m  crv 

ro  ^ 

^  ro  — ( 

CM  to 

0  cn 

m 

CM  CM 

m  ^ 

1 

1 

ro 

to  to 

to  to 

to  to 

tn 

to  to  to 

to  tr> 

to  to 

to 

tn 

to  to 

to  to 

t 

p's 

T-*  f-H 

1-H  1— t  i-H 

-H  f-H 

w'  4 

!■  4 

»—•  l-H 

r-H  l-H 

1 

CSJ  CSJ 

^  CT> 

CM  to 

CT> 

-H  ^  0 

CO  cn 

Lo  ro 

cn  00 

CO  cn 

p-s  CM 

I 

1 

CO 

CT>  CD  to  tn  CO  CO 

r^  0  0 

to  tn  CO  r-^ 

to 

Ps. 

to  r*- 

tn  tn 

1 

CO  CO  CD  00 

OOCOOOCOCOCOOOCO 

COOOtDCOCOtDtDCO 

1 

CD 

<=>  0 

0  0 

CD  0 

CD 

000 

0  0 

0  0 

<D 

0 

0  CD 

0  CD 

CD 

m  ro 

t-H  10 

ro 

to  cn  CO 

CM  CM 

tJD  ^ 

to 

ro  cn 

0  to 

i 

tn 

0  CM 

1—1  ^ 

CM  CT>  C7^ 

CM  0  r^ 

9  i 

CM  cn  CO 

CM 

0  CM 

cn  f-H 

t 

ro 

<3-  «3> 

ro 

ro 

^  'tr  ro 

^  ro 

no 

«a* 

*3“ 

ro  «a’ 

t 

f-H 

0  0 

0  0 

CD  0 

0 

000 

0  0 

CD  CD 

0 

0 

CD  0 

0  0 

CD 

r-H  m  CD 

^  0 

r««. 

«:r  1— C  CM 

CD 

cn 

tn 

tn  1-H 

tn  to 

i 

1 

tn 

0  CO 

CDO^OOCT»1--IC7VCOOO 

cn  CD  cn  cn  p-.  cn 

f-H  00 

1 

na- 

CM  f— t 

»«H  1—1 

«-H  f-H 

f-4  CM  ^ 

f-H  f-H 

t-H 

1—1  1-H 

CM  r-H 

1 

0 

0  CO 

0  c=i 

0  CD 

CD 

CD  CD  CD 

0  CD 

CD  CD 

CD 

CD 

CD  0 

CD  0 

CD 

CO  tn  CO 

CM  »-H 

f-H 

ro 

P--  P^ 

ro  cn 

0 

ro 

CM  cn 

f-H  ro 

1 

f-H 

CTi 

cn  cn 

0>  r«-. 

r-s. 

r--  to  p^ 

P>-  P^ 

P-s.  P-. 

p's 

P^  CO  CO 

CO  cn 

1 

na- 

0  0 

0  0 

0  0 

0 

000 

0  0 

0  0 

0 

0 

0  0 

0  0 

1 

0 

r— ^  r— ^ 

^  fH 

r-H  t-H 

fH 

f-H^ 

rn  CD  CM 

CM  CD 

CM 

Ol  CO  to 

1— (  CM 

in  to 

ro 

P's. 

m  tn 

tn  p's 

1 

1 

CTk 

to  t — 

tOOO^COO^OOOCD^ 

0  p^ 

cn 

p-s  CO 

f-t  CO 

1 

to 

CM  CM 

CM  CM 

CM  ro 

CM 

CM  CM  CM 

ro  CM 

ro  CM 

ro 

CM 

CM  CM 

ro  CM 

1 

0 

0  CO 

CO  CD 

0  CD 

000000 

0  0 

0 

000 

0  0 

0 

m  CO  0  CM 

ro  «:r 

ro 

00  CT>  ro 

cn  00 

to  00  tn 

r-s.  <n 

p's  ^ 

1 

cn 

cn  CO 

to  to 

to  fO 

ro 

f-^  ro  tn 

CM  CM 

ro  ^ 

ro 

ro 

tn  'a- 

no  ^ 

1 

*5r  ^a-  «:r 

«a-  "sr 

"•a*  ^ 

«a-  'a- 

na-  ^ 

i 

^a- 

-- 

f— H 

fH 

t-H 

f-H  f-. 

fH 

«a- 

^  m 

CO  ro 

ro 

ro  ro  ro 

ro  ro 

ro  ro 

ro 

ro 

ro  CM 

CM  CM 

ro 

rs. 

r«-. 

r-. 

p-- 

r>-  r>. 

P-.  p-^ 

Pv. 

p's 

p's  P's 

p's  p-s 

ro 

T—^  w-i 

i-H 

f-H  t-4  f-H 

r-H 

f-H  l-H 

0 

to  to 

to  to 

to  to 

to 

to  to  to 

to  to 

to  to 

to 

to 

to  to 

to  to 

0 

m  m 

m  ro 

ro  ro 

ro 

ro  ro  ro 

ro  ro 

ro  rO 

ro 

ro 

ro  m 

ro  ro 

0 

0000 

000 

000 

0  0 

000 

000 

<D  0 

CD 

CT»  0 

0  CO  CD  CT> 

cn  cn 

cn  CO  CO  cn 

cn  0 

cn  0 

Ha- 

^  LT) 

^  to 

tr>  ^ 

^  'a-  ^ 

fa- 

»a- 

^  tn 

na-  tn 

Ha* 

0  0 

CD  CD 

0  0 

0 

000 

0  0 

0  0 

0 

0 

0  CD 

0  0 

0 

CM  CT>  0  00 

0  0 

0 

f-H  0  ^ 

0  0 

•”* 

CM 

cn 

cn  cn 

CM  0 

cn 

to 

to 

P-^  P^  r— 

P-^  P-. 

to  p^ 

to 

to  to 

P's.  p^ 

r->. 

ps.  p^  r--. 

P^ 

P*-  p-s 

r-s 

p's 

p's  p's 

p's  Ps, 

p's 

f—t  CM 

3 

4 

5 

6 

7 

8 

9 

0 

f-H  CM 

ro  ^ 

tn 

to 

p^  00  cn  0 

LU 

f-H 

^-4 

f-H  f-H 

f-H 

f-H 

f-H  f— t 

1-H  CM 

to 

«  c 

I  00 


A-11 


Table  A2.  (Cont'd) 

Technical/School  Knowledge:  Validities  of  Top  20  Equations  Ranked  According  to  Each  Performance  Index,  at 
the  134-164  Min.  Interval _ _ _ _ _ 

TOP  20  EQUATIONS  RANKED  BY  WHITE-BLACK  SUBGROUP  DIFFERENCE  ESTIMATE  (SUMMARY  ACROSS  9  MOS): 
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Table  A3.  (Cont'd) 

Technical/School  Knowledge:  Validities  of  Top  20  Equations  Ranked  According  to  Each  Performance  Index,  at 
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TOP  20  EQUATIONS  RANKED  BY  VALIDITY  ESTIMATE  (SUMMARY  ACROSS  9  MOS): 
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Table  A4.  (Cont'd) 

On-the-Job  Core  Technical  Proficiency:  Validities  of  Top  20  Equations  Ranked  According  to  Each  Performance 
Index,  at  the  74-104  Min.  Interval _ _ _ 

TOP  20  EQUATIONS  RANKED  BY  DISCRIMINANT  VALIDITY  ESTIMATE  (SUMMARY  ACROSS  9  MOS): 
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Table  A4.  (Cont'd) 

On-the-Job  Core  Technical  Proficiency:  Validities  of  Top  20  Equations  Ranked  According  to  Each  Performance 
Index,  at  the  74-104  Hin.  Interval _ 

TOP  20  EQUATIONS  RANKED  BY  WHITE-BLACK  SUBGROUP  DIFFERENCE  ESTIMATE  (SUMMARY  ACROSS  9  MOS): 
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TOP  20  EQUATIONS  RANKED  BY  MALE-EEMALE  SUBGROUP  DIFFERENCE  ESTIMATE  (SUMMARY  ACROSS  9  MOS): 
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TOP  20  EQUATIONS  RANKED  BY  WHITE-BLACK  SUBGROUP  DIFFERENCE  ESTIMATE  (SUMMARY  ACROSS  9  MOS): 
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Table  A5.  (Cont'd) 

On-the-Job  Core  Technical  Proficiency  Validities  of  Top  20  Equations  Ranked  According  to  Each  Performance  Index,  at 
the  134-164  Hin.  Interval _ _ _ 

TOP  20  EQUATIONS  RANKED  BY  MALE-FEMALE  SUBGROUP  DIEEERENCE  ESTIMATE  (SUMMARY  ACROSS  9  MOS): 
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TOP  20  EQUATIONS  RANKED  BY  BROGDEN  INDEX  ESTIMATE  (SUMMARY  ACROSS  9  MOS): 
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Table  A7.  (Cont'd) 

On-the-Job  Hands-on  Test  Performance:  Validities  of  Top  20  Equations  Ranked  According  to  Each  Performance 
Index,  at  the  74-104  Hin.  Interval _ _ 

TOP  20  EQUATIONS  RANKED  BY  WHITE-BLACK  SUBGROUP  DIFFERENCE  ESTIMATE  (SUMMARY  ACROSS  9  MOS): 
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TOP  20  EQUATIONS  RANKED  BY  WHITE-HISPANIC  SUBGROUP  DIFFERENCE  ESTIMATE  (SUMMARY  ACROSS  9  MOS): 
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TOP  20  EQUATIONS  RANKED  BY  MALE-FEMALE  SUBGROUP  DIFFERENCE  ESTIMATE  (SUMMARY  ACROSS  9  MOS): 
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Table  A8. 

On-the-Job  Hands-on  Test  Performance:  Validities  of  Top  20  Equations  Ranked  According  to  Each  Performance 
Index,  at  the  134-164  Hin.  Interval _ _ _ _ _ 

TOP  20  EQUATIONS  RANKED  BY  VALIOITY  ESTIMATE  (SUMMARY  ACROSS  9  MOS): 
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Table  A8.  (Cont'd) 

On-the-Job  Hands-on  Test  Performance:  Validities  of  Top  20  Equations  Ranked  According  to  Each  Performance 
Index,  at  the  134-164  Hin.  Interval _ _ _ _ 

TOP  20  EQUATIONS  RANKED  BY  BROGDEN  INDEX  ESTIMATE  (SUMMARY  ACROSS  9  MOS): 


4->  <D 
t/1  B 

CD 


a:  fo 

uj  ^  z: 


Q  ^ 

I 

<D 

4-» 
Q_  J= 
IS  : 
rD 

o 

QC 


OsJ  CsJ  CVJ 

z:  z:  s 

<_>  CJ  CJ 


o 

t-h  ^  CNJ  ^  CNJ  CM 


CM  C\J  CM 

z:  z: 

O  O  c-j 


t-HCMCMi-HCMCM»-<CM^CM^ 

_ _ zizisziziziziziiEziz: 

(jC_JC->C_}OCJ(-JC_)OOC_3CJCJC_>C_)CJtJC-)CJt_) 


sz:z:z:z:z:z::ez:z:z:z:^z:z:s:ez:zz 

C->OCJOCJC_>OC»^OCJC->OC->t_JCJC_JC_^Ot,JCJ 

t/^t/^CyOh-CO^--CO</^COoO^—^-“^H;^— 

■sr^yy-yy^agl^T^ZTZrZrZIZIZZZIZIZI 

C_JtJ0CJL->C-)C_><_JC-J0C_>CJ0C->tJ0C-300C,-> 

QCQ:£XOOCUtOa:OZOOOt/)Q-OOOOQ£Q£C2l^Ot: 

Q_a-ca_:E(/iz:Q-Q-coQ_sz:t/izz:Q-Q-Q-z:(^ 

t/ioooot_)CcjcoonCoooo«tC-><_3cot>ooot-Jco 

a^oc^oc>onc50^/^oooooot/^a:ooott 

Oz:2E3czo;<C2::cjQ.Cct:z3:QCO«c<CQ-o 

Q-oooO(/it>oQ^o_cot/^(/ici-Q_t<OGoa.Q_a_a.{^a_ 

oo<ccc<t:t>otncc<3:on(/i<t<ctni/^o^oo<co^ 


o  tn 

Z  C3 
t>0  LTi 

c  c 


on  on  o 

CJ3  :e  cj  z 
on  on  on  on 
<  <  <  <C 


oon*-‘OOac:ononoooooo 

zcuujzzoc,3cai<czzzzz 

onononononcx-onono-ononooc-oon 

CC<C<ton<C<C(/1<CC'<<C’< 


•  X 

on  a> 
o  -a 


ononi— 'onononont/^onononont-^onOLOononcoon 
cacjUJCJC-JOOCJCJOCStULjJCJZC-JtlJtSOtJ 

ononcot/nononononononcnononononononononon 

<c<c<t<cc<cccc<cc<ccccccc<c;c 

coononononononononononononont^t^t^t^^t^ 

<x:ccc<cccc<s:<c<c«a:<:<c<:<ccc<t: 

onononooonononononc/iononoot/nt/iononononon 

c<c<ccc<c<c<cc<cccc<!«tc«a:c<i<c 

:s33:3:^^3::s:s::s3::s3c^:sis3::s3::s 
ononoit/ionononononononononononLOonc^cgc^ 
^  ^  ^  ^  ^  ^  ^  ^  ^  ^  ^  ^  ^  ^  ^ 
occ£c2o£acccc2ccc^cccriaio^ococa:acaiccc£: 

cc<a:cc<c<:c<<<<c<t<<c<<c<<c<c 

ononoococoonononononononoot^t^t^(^<^<^<^ 
^1— <•^00*— tOO*-HOOOO 


on  o  CM  CM  t—t  o  oO  '>o  CO  un  cty  co  ^  •— *  to 

tDtototDtounvDununtDununto^ununtotDunun 


^mcotOOOtotDCOto«=rcj5^rocM<oroocMCOO 
^CTYtOr^CMiOtO^tO^COCMrOUnOCOCTlCTT^CM 
p>.r^i —  cOCOP^tOCOCOCOCOP^COCOCOr^c^c^COtO 

oooooooooooooooooooo 

mCMP^CTT'«t'«-nCO'-^*^tDrO^COO*-HiOrO'^r>.CD 
CM«— (CTYco^p^oooo'^i’cnr^or^unmt^ 
p>.r>.rv.c0r^r^tDr^00C0r^f^C0r^vor^tOtDC0r^ 

OCDOOOOOOOCDOOOOOOOOOO 

or^r^to^<30COtncT>untotocMununoococT>CT»C3n 

CM<— touncMO^or^cMOOCTioO'-o^i-O'^rcn^ 

^^^^Ln«:rpoun^ir)^^'d-^ro<5t-cooo^«:r 

oooooooooooooooooooo 

r-itD^ir)t0t0O^Cn»-H^00CM<OOr^r^C0C7^00O> 

rocMunooto«d-otnr«..r^unr««.oOtooOCTiCMor^co 

CP>CT>O^OOOOCT>CT>COCOCOCOCJTCOCOCOCOCTTChCOC7^ 

OC300000<0000000000000 

CM^tOOO'— ‘ChOCOmcOCTTr-^CMrv.GnP^tO'^P'.CO 
unun'^r«..p^^ior^tor^toco<ou3tDCT>ro«^toto 
to  to  to  totototototototountotototototototn 

oooooooooooooooooooo 

r>*.onoocna^rocX5P^CMOtOtototocMr^Otocz>to 
OOOOOYrOmCTkt— <^ununCT>^tO»:rCM00'=f»-<tOCTY 
(«— (t— •»-HCM»-Hr-<CM«— tCMCM.-Hr-( 


I  ^mcMf-Hr-H»-(r-(T-(oooooCT>c7>c7>c7Tcy>cncTT 
I  OOOOOOOOOOOOOCJSC3nCT»C7TCJ>CT>Cy> 

I  CMCMCMCMCMCMCMCMCMCMCMCMCMt— c,— ( 


C3n 

CO 


s_  >>  I  ^pncMto^^'^rocopOfOpncMtncM^-HCMCMLnpo 
04J  I 

t/l-O  I  oooooooooooooooooooo 


1  tor^«:r^CMrotorocM<MP^toOi-HOOromunCTYr^ 
1  o»cr»o^ooo^o>ooocTYO^ooo^oo^o^CT>cT> 
I  ooOt^«— <00»— •«— It— tOO»— iT~tO»— 'OOOO 


•o  c  I  tr>rocMr-H^ci-CMCT^Lnr^cOtnCMtoi^'^cncM«=j-tD 
03  t  OOOOCTTO«-'CTiCJ>CT>OOC3nCTiO<OOt-HanO 
< —  a>  I  unur)tjr>Lr>«:runtn«5r'=r«:3-tntn^<^'tntnuntrj<rtn 


<CMC*n<rtOtOP^COCDO«— •CMCO^tOtOr^OOCJ'O  I 


t  CM  I  on 


A-47 


Table  A8.  (Cont'd) 

On-the-Job  Hands-on  Test  Performance;  Validities  of  Top  20  Equations  Ranked  According  to  Each  Performance 
Index,  at  the  134-164  Min.  Interval _ 

TOP  20  EQUATIONS  RANKED  BY  WHITE-BLACK  SUBGROUP  DIFFERENCE  ESTIMATE  (SUMMARY  ACROSS  9  MOS): 
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Table  A8.  (Cont'd) 

On-the~Job  Hands-on  Test  Performance:  Validities  of  Top  20  Equations  Ranked  According  to  Each  Performance 
Index,  at  the  134-164  Min,  Interval _ 

TOP  20  EQUATIONS  RANKED  BY  MALE-FEMALE  SUBGROUP  DIFFERENCE  ESTIMATE  (SUMMARY  ACROSS  9  MOS): 
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Table  A9.  (Cont'd) 

On-the-Job  Hands-on  Test  Performance:  Validities  of  Top  20  Equations  Ranked  According  to  Each  Performance 
Index,  at  the  194-224  Interval _ 

TOP  20  EQUATIONS  RANKED  BY  BROGDEN  INDEX  ESTIMATE  (SUMMARY  ACROSS  9  MOS): 
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Table  A9.  (Cont'd) 

On-the-Job  Hands-on  Test  Performance:  Validities  of  Top  20  Equations  Ranked  According  to  Each  Performance 
Index,  at  the  194-224  Interval _ 

TOP  20  EQUATIONS  RANKED  BY  WHITE-HISPANIC  SUBGROUP  DIFFERENCE  ESTIMATE  (SUMMARY  ACROSS  9  MOS): 
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TOP  20  EQUATIONS  RANKED  BY  MALE-FEMALE  SUBGROUP  DIFFERENCE  ESTIMATE  (SUMMARY  ACROSS  9  MOS): 
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Appendix  B 


LVII  SOLDIERS  MEETING  PREDICTOR/CRITERION 
SETWISE  DELECTIOM  DATA  REQURIEMEMTS  FOR 
VALIDATION  OF  EXPERIMENTAL  BATTERY  PREDICTOR  COMPOSITES 
AGAINST  PERFORMANCE  CRITERIA 


Table  B.l 


Soldiers  in  LVII  Meeting  Predictor/Criterion  Setwise  Deletion  Data 
Requirements  for  Validation  of  ASVAB  Operational  Scores  and  Spatial,  Computer, 
JOB,  ABLE,  and  AVOICE  Experimental  Battery  Predictor  Composites  Against  Core 
Technical  Proficiency  Factor  by  MOS _ _ _ _ _ 

Predictor  Sets _ _ _ 


MOS 

ASVAB 

Spatial 

Computer 

JOB 

ABLE 

AVOICE 

IIB 

333 

322 

112 

301 

297 

309 

13B 

170 

165 

152 

159 

148 

156 

19K 

156 

130 

130 

122 

123 

129 

63B 

169 

147 

139 

136 

132 

140 

71L 

147 

115 

104 

105 

109 

102 

88M® 

84 

56 

54 

51 

52 

53 

91A 

205 

191 

174 

185 

165 

183 

95B 

160 

149 

140 

142 

133 

145 

Total 

1,424 

1,275 

1,005 

1,201 

1,159 

1,217 

“mOS  88M  not  included  in  LVII  validity  analyses. 


Table  B.2 

Soldiers  in  LVII  Meeting  Predictor/Criterion  Setwise  Deletion  Data 
Requirements  for  Validation  of  ASVAB  Operational  Scores  and  Spatial,  Computer, 
JOB,  ABLE,  and  AVOICE  Experimental  Battery  Predictor  Composites  Against 
General  Soldiering  Proficiency  Factor  by  MOS _ _ _ 

Predictor  Sets _ _ _ 


MOS 

ASVAB 

Spatial 

Computer 

JOB 

ABLE 

AVOICE 

IIB^ 

-- 

13B 

170 

165 

152 

159 

148 

156 

19K 

156 

130 

130 

122 

123 

129 

63B 

169 

147 

139 

136 

132 

140 

71L 

147 

115 

104 

105 

109 

102 

88m‘’ 

84 

56 

54 

51 

52 

53 

91A 

205 

191 

174 

185 

165 

183 

95B 

160 

149 

140 

142 

133 

145 

Total 

1,091 

953 

893 

900 

862 

908 

*H0S  IIB  not  included  in  LVII  validity  analyses  for  General  Soldiering  Proficiency. 
‘’MOS  88H  not  included  in  LVII  validity  analyses. 


B-1 


Table  B.3 


Soldiers  in  LVII  Meeting  Predictor/Criterion  Setwise  Deletion  Data 
Regu i rements  for  Validation  of  ASVAB  Operational  Scores  and  Spatial,  Computer, 
JOB,  ABLE,  and  AVOICE  Experimental  Battery  Predictor  Composites  Against 
Achievement  and  Effort  Factor  by  MOS _ _ _ 


Predictor  Sets 


MOS 

ASVAB 

Spatial 

Computer 

JOB 

ABLE 

AVOICE 

IIB 

310 

300 

105 

280 

276 

286 

13B 

157 

152 

140 

143 

136 

143 

19K 

138 

116 

115 

no 

112 

115 

63B 

183 

157 

148 

145 

140 

149 

71L 

140 

109 

99 

100 

103 

96 

88M'‘ 

80 

52 

50 

48 

49 

50 

91A 

179 

165 

151 

158 

144 

157 

95B 

146 

137 

129 

130 

121 

133 

Total 

1,333 

1,188 

937 

1,114 

1,081 

1,129 

*M0S  88M  not 

included  in  LVII 

validity  analyses. 

Table  B.4 

Soldiers  in  LVII  Meeting  Predictor/Criterion  Setwise  Deletion  Data 
Requirements  for  Validation  of  ASVAB  Operational  Scores  and  Spatial,  Computer, 
JOB,  ABLE,  and  AVOICE  Experimental  Battery  Predictor  Composites  Against 
Personal  Discipline  Factor  by  MOS _ _ _ _ 


MOS 

Predictor 

Sets 

ASVAB 

Spatial 

Computer 

JOB 

ABLE 

AVOICE 

IIB 

320 

310 

109 

290 

286 

296 

13B 

169 

164 

152 

155 

147 

154 

19K 

143 

120 

120 

113 

116 

119 

63B 

184 

158 

149 

146 

141 

150 

71L 

144 

111 

101 

102 

105 

98 

88M^ 

83 

55 

53 

50 

52 

52 

91A 

205 

190 

173 

182 

164 

181 

95B 

161 

150 

141 

143 

134 

146 

Total 

1,409 

1,258 

998 

1,181 

1,145 

1,196 

*M0S  88M  not  included  in  LVII  validity  analyses. 
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Table  B.5 


Soldiers  in  LVII  Meeting  Predictor/Criterion  Setwise  Deletion  Data 
Requirements  for  Validation  of  ASVAB  Operational  Scores  and  Spatial,  Computer, 
JOB,  ABLE,  and  AVOICE  Experimental  Battery  Predictor  Composites  Against 
Physical  Fitness  and  Military  Bearing  Factor  by  MOS _ _ _ 

_ Predictor  Sets _ _ _ 


MOS 

ASVAB 

Spatial 

Computer 

JOB 

ABLE 

AVOICE 

IIB 

309 

299 

107 

281 

278 

286 

13B 

162 

157 

145 

148 

141 

147 

19K 

137 

117 

117 

no 

113 

116 

63B 

177 

151 

142 

140 

134 

143 

71L 

143 

no 

100 

101 

104 

97 

88M^ 

81 

54 

52 

49 

51 

51 

91A 

194 

179 

164 

172 

154 

171 

95B 

158 

147 

138 

141 

131 

143 

Total 

1,361 

1,214 

965 

1,142 

1,106 

1,154 

‘mOS  88M  not  included  in  LVII  validity  analyses. 


Table  B.6 

Soldiers  in  LVII  Meeting  Predictor/Criterion  Setwise  Deletion  Data 
Requirements  for  Validation  of  ASVAB  Operational  Scores  and  Spatial,  Computer, 
JOB,  ABLE,  and  AVOICE  Experimental  Battery  Predictor  Composites  Against 
Leadership  Factor  by  MOS _ 


Predictor  Sets 


MOS 

ASVAB 

Spatial 

r  1  uu  1  uw  1 

Computer 

JOB 

ABLE 

AVOICE 

llB 

298 

290 

103 

270 

266 

277 

13B 

131 

127 

117 

122 

no 

119 

19K 

114 

99 

100 

93 

98 

98 

63B 

169 

146 

137 

136 

130 

139 

71L 

130 

104 

95 

96 

99 

94 

88M^ 

69 

48 

45 

44 

45 

46 

91A 

183 

170 

158 

165 

153 

164 

95B 

146 

136 

126 

130 

122 

132 

Total 

1,240 

1,120 

881 

1,056 

1,023 

1,069 

“mOS  88M  not  included  in  LVII  validity  analyses. 
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Table  B.7 


Soldiers  in  LVII  Meeting  Predictor/Critenon  Setwise  Deletion  Data 
Requirements  for  Validation  of  ASVAB  Operational  Scores  and  Spatial,  Computer, 
JOB,  ABLE,  and  AVOICE  Experimental  Battery  Predictor  Composites  Against  Hands- 


On  total  Score  bv  MOS  - - - 

MOS 

Predictor  Sets 

ASVAB 

Spatial 

Computer 

JOB 

ABLE 

AVOICE 

IIB 

333 

322 

112 

301 

297 

309 

13B 

171 

166 

153 

160 

148 

157 

19K 

156 

130 

130 

122 

123 

129 

63B 

171 

149 

141 

138 

134 

142 

71L 

149 

116 

105 

106 

no 

103 

88M^ 

84 

56 

54 

51 

52 

53 

91A 

206 

192 

175 

186 

166 

184 

95B 

160 

149 

140 

142 

133 

145 

Total 

1,430 

1,280 

1,010 

1,206 

1,163 

1,222 

“mOS  88M  not  included  in  LVII  validity  analyses. 

Table  B.8 

Soldiers  in  LVII  Meeting  Predictor/Criterion  Setwise  Deletion  Data 

Requirements  for  Validation  of  ASVAB  Operational  Scores  and  Spatial,  Computer, 
JOB,  ABLE,  and  AVOICE  Experimental  Battery  Predictor  Composites  Against  JoD 
I^nnwledne  Total  Score  bv  MOS 

MOS 

Predictor  Sets 

ASVAB 

Spatial 

Computer 

JOB 

ABLE 

AVOICE 

IIB 

341 

330 

115 

309 

305 

316 

13B 

178 

173 

160 

164 

156 

163 

19K 

164 

138 

137 

130 

131 

137 

63B 

186 

160 

151 

148 

143 

152 

71L 

149 

116 

105 

106 

no 

103 

88M^ 

85 

57 

55 

52 

53 

54 

91A 

216 

201 

183 

193 

173 

192 

95B 

164 

153 

144 

146 

137 

149 

Total 

1,483 

1,328 

1,050 

1,248 

1,208 

1,266 

“mOS  88M  not  included  in  LVII  validity  analyses. 
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Appendix  C 

ARMY  JOB  SATISFACTION  QUESTIONNAIRE  (AJSQ) 
LVII  Version 


Note  the  following  changes  from  the  LVI/CVII  version  of  the  AJSQ: 
Section  I 

•  Items  13,  16,  and  23  were  added. 

Section  II 


•  The  following  four  items  were  dropped: 

*  When  you  decided  to  enlist,  how  sure  were  you  about 
what  training  you  wanted? 

*  In  high  school,  what  education  program  were  you  in? 

*  Has  your  father,  mother,  or  guardian  ever  served,  or 
is  (s)he  now  serving  in  the  military? 

*  In  what  MOS  do  you  now  work  ( i.e.  ,  your  current  duty 
assignment)? 

•  Items  33  through  57  were  added. 
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Nam* 


SS#_ 


Army  Job  Satisfaction  Questionnaire 


Section  1; 

This  section  asks  questions  atxiut  your  supervisors,  your  co-workers,  and  your  job/career.  We 
would  like  to  know  how  you  teel  about  eacn  of  these  aspects  of  Army  life.  Please  read  eachi 
statement  carefully  and  deade  how  you  feel  about  it  Mark  the  box  that  corresponds  to  youri 
level  of  satisfaction  with  the  aspect  of  Army  life  descnbed  by  the  statement  For  example, 
mark  ‘Very  Satisfied*  if  it  is  much  better  than  you  hoped  it  would  be.  Or,  you  might  mark  ‘Very  i 
Dissatisfied*  if  it  is  much  poorer  than  you  would  like  it  to  be. 

Mark  vour  answers  by  putting  an  ‘X*  over  the  aporoonate  box  like  this:  IS _ 


A.  Supenlslon:  How  sattsfled  are  you  with... 

1.  Your  immediate  supervisor's  capaoilities? _ 

2.  The  way  yoirimmediata  supervisor  handles  soldiers  in  yoir  unit?. 


.□  □ 


3. 


Overall,  how  satisfied  are  you  with  the  quality  of  supervision  you  are  currently 
tecetwig?  . . . . . . . .  ,  . . .  . 


o  □ 


B.  Co-WOrken:  How  sadsflod  are  you  with... 

4.  The  amount  of  respect  you  get  from  the  soldiers  in  your  unit? - 

5.  The  way  soldiers  in  your  unit  work  together  to  finish  a  job? - 

6.  Overall,  how  satisfied  are  you  with  your  co-workers  in  the  Army? 


o  □ 
.□  □ 
o  □ 


c. 


Promodora:  How  aataaed  are  you  with... 

7.  Your  chances  for  promotiors  in  yotxcurent  duty  MOS? - 

8.  Vftiif  rhanro*  nf  gotting  ahoart  in  Iho  Army? 

9.  Overall,  how  satisfied  are  you  with  your  opportunities  tor  promotion?. 


o  □ 
.□  □ 
o  □ 


0.  Pay;  How  satlsfled  are  you  with  the  way  yotf  Army  pay... 

10.  Meets  your  personal  and  family  needs? _ 

11.  Covers  the  cost  of  living  wnere  you  are  currently  assigned?. 

12.  Overall,  how  satisfied  are  you  with  your  salary? 


,□  □ 
.□  □ 
.□  □ 


E 


Work  (Your  current  duty  assignment):  How  satisfied  are  you  with... 

13.  Thptypanfwvfcynnrtn7 

14.  Your  opportunity  to  do  important  things  on  the  job?— — — 

1&  Your  opportunity  to  use  your  abilities,  experience,  and  training? _ 

16.  YNr<yptMirtlyinf»iintPnMaifvjvi^ 

17.  The  amount  Of  Challenge  myotx  work? _ 

18.  Overall,  how  satisfied  are  you  with  your  current  duty  assignment? 


,□  □ 
.□  □ 
.□  □ 
.□  □ 
.□  □ 
.□  □ 


I 


□  □□ 
□  □□ 


□  □□ 


□  □□ 
□  □□ 
□  □□ 

□  □□ 
□  □□ 
□  □□ 

□  □□ 
□  □□ 


□  □□ 
□  □□ 
□  □□ 
□  □□ 
□  □□ 
□  □□ 


Survay  Approval  Authonty;  U.S.  Army  Hei^onnei  imegraiion  Gommana 
Survay  Control  Numben  ATNC-AO-91-52 
RCS:  MILPC-3 
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F.ThaArnryasancarganjzailon:  Howsateaadareyouwtth... 

19.  The  way  the  Army  treats  its  solders? - - - 

20.  Your  opportunity  to  spend  time  with  your  family? - 

21.  TheanKmodeisuBiiiieyDuhaw? - — 

22.  In  genera,  how  satisfied  are  you  with  all  aspects  of  Amty  We?. 


.:2‘ 

o  □  □  □  □ 
.□□□□□ 
.□□□□□ 
.□□□□□ 


23.  /uiiMngsconsktaBd,hawdoyouf8ef  aboutyourjobsttuattoninosotthetinw? 

Best  imaginable 
□  Highly  favorable 
Satisfactory 
Q  Neutral 

Q  Poor 

O  Very  bad 

Q  Worst  naginabie 


Sectionit: 

The  questions  in  this  section  are  related  to  your  reasons  for  enlisting,  your  future  plans,  and 

the  factors  that  may  bo  influencing  these  plans.  Please  read  each  question  carefully  and 
answer  as  best  youcan.  Marfc  your  answers  by  putting  an ‘X*  over  the  aporoonate  box. 


lybilt  the  beat  Indicating  tftefiiporianca  of  each  reason  for  enlisting; 

24.  To  serve  my  country  - - - 

25.  Togetttaredinaslal - - - ; - - - - 

26.  Money  for  education  or  training  (e.g.,  college  or  vocational,  technical,  or  business 

school) - — - - - 

27.  To  be  a  solder -  - - - 

28.  The  physical  training  and  challenge. - — - 

29.  To  team  to  be  a  responsible  mature  person  . . — 


^  ^  c?  It? 

.□□□□□ 

.□□□□□ 

.□□□□□ 

.□□□□□ 

.□□□□□ 


30.  Would  you  prefer  wortdng  in  a  dfferent  MOS? 

Q  No 
O  Yes 
Q|  Not  sure 


31.  If  you  could  reenlist  tomorrow,  which  MOS  would  you 
select? 

Q  My  Primary  (AIT  completed)  MOS 

Q  My  Secondary  (AIT  completed)  MOS 

Q  A  Duty  MOS  other  than  my  Primary  or  Secondary 


32.  wnat  00  you  tninK  you  w«l  oo  ansr  ms 
eniisimere? 

Leave  tne  Army  to  lind  civiiian  emptoyment 

Leave  tfie  Army  to  attend  college 

O  Leave  the  Army  for  civilian  vocational/ 
technical  education 

d  R.eenlist 
O  I  do  not  know 


M.  wnaiiSEneieveionnoraieiriyaiuniir  nyoutnom 
somefxace  else,  answer  this  question  tor  the 
place  wnere  you  wxK. 

O  Very  low 

O  Low 

O  Moderate 

□  High 

Cli  Very  high 


34.  Overall,  how  fairty  has  the  Army  treated  you? 


O  Very  fairly 
Somewhat  fairly 
Q  Neither  fairly  nor  unfairly 
O  Somewhat  unfairly 
Very  unfairly 


31  Oo  the  recant  devetopments  iri  the  Middle  East  (i.e..  Operation  Desert  Shieid/Stoim) 

make  you  more  or  less  interested  in  serving  in  the  Army?  ■—  - - 

31  Oo  recent  developments  in  Eastern  Europe  (e4^  the  taH  ot  the  communist 

governments)  make  you  more  or  less  interested  in  serwng  in  the  Army? - - 

37.  Do  Ekely  reductions  in  the  size  of  the  U.S.  Army  make  you  more  or  less  interested  in 

aoivinginiha  Amy?  - - 

31  Does  the  current  state  of  the  LI.S.  economy  make  you  more  or  less  interested  in 

<ain^inihft  Arniy?  - — - 


1  # 


i  /  # 
#  i 


1 

S 


i 


.□  □  □  □ 


□  □  □ 


39.  What  effect  if  any,  have  your  Army  expenences  had  on  the  development  of  specific 
job  knowledge,  skills,  and  abilities  that  wilt  help  you  obtain  a  civilian  job? 

41  What  effect  if  any,  have  your  Army  expenences  had  on  the  development 

of  personal  characteristics  and  attitudes  that  will  help  you  obtain  a  civilian  job? — 


o  □  □ 
.□  □  □ 


□  □ 
□  □ 
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Are  you  nm  concaTjed  abort  tnese  mings  today  tnan  you 

41.  Your  long-tenn  opportunities  in  the  Aniiy  — 

42.  The  kind  ot  vwO  you  migtt  lave  O  do  in  the  Army - — 

41  The  kind  ot  work  you  wi  go  into  when  you  leave  the  Aiiny - 

44.  Whether  you  would  be  able  to  get  a  civiHan  job  quickly  if  you  had  to  . 

41  The  financial  buden  on  you  and  yotr  family  should  you  have  to  leave  (he 

AnnyuBasciBdly —  1 1 " 

41  Problems  associated  with  deployment  (e.g.,  separation  from  family,  childcare 
arrangements,  financial  burdens) - - 


#  #1 


l/s  / 

!■##  s 

'i/  / 


Qi  <a  G 


.□□□□□ 

.□□□□□ 

.□□□□□ 


How  much  better  or  worse  do  you  think  these  aspects  woiid  be  for  you  in  civilian  kfe  than 
they  are  for  you  In  the  Anny? 

47.  Your  income - - — 

41  Yotr  abity  to  dewloo  community  ties - - — > 

41  Support  services  tor  families  ,  ■  ■  — 

9L  Quality  of  your  co^wxkers  —  . 

51.  Personal  freedom - - 

51  Credt  for  doing  a  good  job  . . 

53.  Type  of  work  you  fin 

54.  YotjpromoMnoppomniies _  _ 

51  Your  job  secunty  — 

51  Yoix  treatment  by  supervrsors . — 

57.  Your  feefngs  of  commitment  to  yoir  employer  . . . 


i  « 


£ 


Q. 


.□  □  □  □  a 
.□□□□□ 
.□□□□a. 
.□□□□□ 
.□□□□a. 
.□□□□□ 
.□□□□□ 

.□□□□□ 
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